Topic: Pianoteq benchmarks -- performance across various hardware

I just received my tiny/fanless computer to replace my old/noisy laptop.  It uses a i5-4220Y (Haswell) processor -- dual core with hyperthreading, 1.6ghz with turbo-boost to 2.0ghz.  It supposedly has a TDP of 11.5W but when I plugged in my electric meter, I see it ranges from 8W @ 600mhz idle to 22W @ 2.0ghz max load turbo.  I then plugged in my Celeron 2955U laptop (15W rating) to see what kind of electric usage it has with the screen off -- 8W @ 800mhz idle to 14W @ 1.4ghz max load.  Excel isn't needed to figure out the power profile -- Haswell U/Y chips use about 1W per 100mhz of operation.  Hence, a 1.6ghz Y chip (TDP 11.5W) probably doesn't uses any less power than a 1.6ghz U chip (TDP 15W).  It's mostly just Intel marketing with some thermal throttling thrown in in keep the average consumption lower.

Beyond the misleading Intel specifications, this box is working out pretty good.  It's small and makes no sound at all which is wonderful when for working at night when everything is absolutely quiet.  The entire case is an aluminum block heatsink.  Ambient temperatures are 25C now -- sensor data shows the processor at 40C during light loads, 50C under max loads.  When it hits 50C, the case is like a slightly hot sauna rock that you can still hold for 5-10 seconds at a time.  It did overheat once when I left it running something nonstop for an hour.

As I am working in China, I was able to order this machine with 4GB RAM + 128GB SSD from Taobao (Alibaba) for a whopping total of 1190 CNY ($187 USD).

  https://item.taobao.com/item.htm?spm=a2...t=5#detail

If you are in the US, you can order from AliExpress for $239 USD which isn't that bad of a premium for a product shipped from China.

  http://www.aliexpress.com/store/product...46485.html




Now onto Pianoteq performance.  With a few more boxes now available for testing, I re-ran my Pianoteq benchmark script and normalized it to matchup with the Pianoteq performance index number.  For the benchmark, I instead held the foot pedals down and pounded the keyboard in order to generate as many simultaneous voices as possible because a regular MIDI file (real music) does not have enough polyphony to generate consistent results when you go beyond 2 cores.

Here are peak numbers for each machine I tested.  Where there is a T in the ghz column, that means that CPU has the ability to turbo-boost beyond the rated speeds.  In the cores column, the +HT means extra virtual cores via Intel's hyperthreading.

Score Processor     Arch         ghz  Cores
----- ------------- ------------ ---- -----
60.8  i7-4790K      Haswell      4.0T 4+HT
51.2  i5-3450S      Ivy Bridge   2.8T 4
45.4  i5-2500       Sandy Bridge 3.3T 4
40.0  FX-8350       BD Vishera   4.1T 8
39.4  i7-920        Nehalem      2.7T 4+HT
37.6  A10-6700      BD Richland  3.7T 4
35.3  Core2 E8400   C2 Wolfdale  3.0  2
32.7  X2-250        Ph2 Regor    3.0  2
31.7  i5-430M       Nehalem-C    2.2T 2+HT
31.2  FX-8120       BD Bulldozer 3.1T 8
31.2  Phenom2 1100T Ph2 Thuban   3.2T 6
29.5  Phenom2 940   Ph2 Deneb    3.0  4
29.4  i3-3227U      Ivy Bridge   1.9  2+HT
29.2  i5-4220Y      Haswell      1.6T 2+HT
27.7  Core2 E6600   C2 Conroe    2.4  2
25.8  Phenom2 940   Ph2 Deneb    3.2  4
24.3  Celeron 2955U Haswell      1.4  2
23.0  X4-645        Ph2 Propus   3.2  4
23.7  Avoton C2750  Bay Trail    2.4T 8
16.9  A4-5000       Kabini       1.5  4
14.6  Pentium T2310 C2 Merom-2M  1.5  2
 5.2  E1-1200       Bobcat       1.4  2
 5.0  Atom N450     Atom         1.7  1+HT
 4.3  Mobile P4     Northwood    2.8  1

From using Pianoteq on a few of these machines, here's what some of these numbers roughly correlate to:
   *  5 = 22khz external, 11khz internal, 16 voices, ~8ms latency
   * 10 = 22/11/32/8
   * 15 = 22/11/64/8
   * 20 = 22/11/256/8
   * 25 = 44/22/256/8
   * 30 = 96/48/256/8




For the next step, I clocked every machine to a near equal frequency of 1.6ghz.  With these numbers, you can then extrapolate the performance of the entire range of a processor family.

Score Arch           Cores
----- -------------- -----
25.4  Haswell        4+HT
27.1  Haswell        2+HT
24.7  Haswell        2
25.0  Ivy Bridge     2+HT
23.4  Sandy Bridge   4
23.3  Nehalem-C      2+HT
23.0  Nehalem        4+HT
19.5  Core2 Wolfdate 2
18.7  Core2 Conroe   2
17.7  Kabini         4
16.0  Bay Trail      4-8
15.6  BD/Richland    4
15.3  Bay Trail      2
15.1  BD/Bulldozer   8
14.8  BD/Vishera     8
 5.7  Bobcat         2
 5.0  Atom           1+HT

Clock-for-clock, the top architecture tested was Intel Haswell.  (I don't have any Broadwell machines yet.)  The 4ghz+ Haswell downclocked ends up being slower than the chips who's native speeds are ~1.6 as high ghz chips often have internal optimizations to favor higher ghz over low.  The older iCore chips (Nehalem, Sandy Bridge, Ivy Bridge) come in close behind so they still will work pretty good for Pianoteq (at the cost of power & heat).




To further expore the effects of multi-core and hyperthreading, I picked a few chips, re-ran the tests using the lowest speed possible and then turned off the cores one-by-one.

Score Cores
----- -----

Kabini / A4-5000 800mhz
 9.9  4
 9.7  3
 9.6  2
 5.6  1

Bay Trail / C2750 1.3ghz
13.2  4
13.2  3
12.7  2
 8.0  1

Haswell / i5-4220Y 600mhz
10.4  2+HT
10.1  2   
 8.3  1+HT
 5.7  1   

Ivy Bridge / i3-3227U 800mhz
12.7  2+HT
12.7  2
10.6  1+HT
 7.3  1

Nehalem-C / i5-430M 1.2ghz
17.9  2+HT
17.6  2    
12.5  1+HT
10.3  1    

Going from 1 core to 2 cores across all architectures (except for Bay Trail) adds a 70-75% improvement in performance:

  Bay Trail:  +60%
  Kabini:     +70%
  Nehalem-C:  +70%
  Ivy Bridge: +75%
  Haswell:    +75%

By comparison, going from 1 core to 1 + 1HT (virtual core) is +45% which while less than a full core but it is still much better a poke in the eye:

  Nehalem-C:  +45%
  Ivy Bridge: +45%
  Haswell:    +45%

 
After 2 cores, adding more cores (both real & virtual) adds a tiny bit of performance although the effect increases as with slower architectures/clocks.

  Bay Trail:  +4%
  Kabini:     +3%
  Nehalem-C:  +2%
  Ivy Bridge: +0%
  Haswell:    +3%



Speaking of Bay Trail, these processors come up often for use with Pianoteq due to their low cost/power and ubiquity in tablets/compute sticks.  For that reason, I ran a few more variations on the Avoton C2750 to simulate other Bay Trail processors running Pianoteq.

Score ghz Cores Equivalent Processor
----- --- ----- --------------------
21.9  2.4 4     Pentium N3540/J2900
21.0  2.4 2     Celeron N2820
19.3  2.0 4     Celeron N2920/J1800
18.4  2.0 2     Celeron N2810
16.0  1.6 4     Pentium N3700, Celeron N2920, Atom Z3975/Z8700
15.3  1.6 2     Celeron N3150/N3050/N2806, Atom Z3460
13.2  1.3 4     Atom Z3740/Z3530/Z8300/Z8700
12.7  1.3 2     Atom Z3680

In terms of matching up theoretical Bay Trail products, I ignored the rated turbo speeds.  If you have a fanless tablet or compute stick, turbo is not going to kick in often due to thermal limitations -- especially since Pianoteq will use 2.5 cores on a slow architecture.

Re: Pianoteq benchmarks -- performance across various hardware

Wow, you've had QUITE some time on your hands...

These numbers are very useful and interesting. Thanks

Hard work and guts!

Re: Pianoteq benchmarks -- performance across various hardware

Great work, and very useful also. This can be a rough guide for a lot of us in the future!

I wonder if your overall readings are a little on the low side--only because I tried your rough test of holding the pedal down and swiping back and forth across the Pianoteq virtual keyboard and got a surprisingly high number on my four year old desktop: a peak of 45-48, with peak polyphony of around 230, on an Intel I7 x980 3.33HGZ (Pianoteq hieroglyphics on the processor: m12/s2/f6 x12). Some of the processors you list are definitely faster than this one here.

Re: Pianoteq benchmarks -- performance across various hardware

NormB wrote:

I wonder if your overall readings are a little on the low side--only because I tried your rough test of holding the pedal down and swiping back and forth across the Pianoteq virtual keyboard and got a surprisingly high number on my four year old desktop: a peak of 45-48, with peak polyphony of around 230, on an Intel I7 x980 3.33HGZ (Pianoteq hieroglyphics on the processor: m12/s2/f6 x12). Some of the processors you list are definitely faster than this one here.

My goal was to "try" to get close to Pianoteq's index.  But since my logic and that logic is different, only in the middle do we match.  The scale at the top and bottom will be different.

Re: Pianoteq benchmarks -- performance across various hardware

Respect, Mossy, much work from you to correlate Modartt's "Performance Index" with some CPU specs!

Good simulation of reality for example:

Mossy wrote:

Score ghz Cores Equivalent Processor
----- --- ----- --------------------
21.9  2.4 4     Pentium N3540/J2900
21.0  2.4 2     Celeron N2820

... as my Celeron N2930 has a performance index of 21 in my real PTQ-environment.

But the following seems to be a bit pessimistic maybe ...

Mossy wrote:

From using Pianoteq on a few of these machines, here's what some of these numbers roughly correlate to:
   *  5 = 22khz external, 11khz internal, 16 voices, ~8ms latency
   * 10 = 22/11/32/8
   * 15 = 22/11/64/8
   * 20 = 22/11/256/8
   * 25 = 44/22/256/8
   * 30 = 96/48/256/8

..., as I can easily play with 21 = 48/48/64/1.3 in your diction.

Nice little fanless PC you bought! Could you get Linux running on it? Can UEFI and "secureboot" be reset to legacy BIOS? Which audiocodec is onboard? Many questions, sorry.

Thanks anyway

Last edited by groovy (19-12-2015 17:34)

Re: Pianoteq benchmarks -- performance across various hardware

groovy wrote:
Mossy wrote:

From using Pianoteq on a few of these machines, here's what some of these numbers roughly correlate to:
   *  5 = 22khz external, 11khz internal, 16 voices, ~8ms latency
   * 10 = 22/11/32/8
   * 15 = 22/11/64/8
   * 20 = 22/11/256/8
   * 25 = 44/22/256/8
   * 30 = 96/48/256/8

I guess it would be more accurate to say the above range correlates to "worse" case settings.  When I fired up the torture test MIDI file, I heard a few dropped notes so I based it on that.  For music we ordinary humans play, you can probably double the settings.  Using a "mere" 100+ polyphony MIDI file to test, the best Pianoteq architecture actually was AMD Kabini.

groovy wrote:

Nice little fanless PC you bought! Could you get Linux running on it? Can UEFI and "secureboot" be reset to legacy BIOS? Which audiocodec is onboard? Many questions, sorry.

I am running Linux Mint 17.2 on this fanless PC.  I certainly would not have tried to do the benchmarks under Windows -- just too hard to automate everything.

Now that you mentioned Legacy vs UEFI, that probably explains why my first install worked using EXT2 /boot but then after fiddling around with settings (and choosing BIOS Restore Defaults), I had to reinstall with UEFI /boot.

Here's what Linux reports as the devices:

00:02.0 VGA compatible controller: Intel Corporation Device 0a1e (rev 0b)
00:03.0 Audio device: Intel Corporation Haswell-ULT HD Audio Controller (rev 0b)
00:1b.0 Audio device: Intel Corporation Lynx Point-LP HD Audio Controller (rev 04)
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
03:00.0 Network controller: Broadcom Corporation BCM43224 802.11a/b/g/n (rev 01)

Re: Pianoteq benchmarks -- performance across various hardware

Here's my box at Amazon:

http://www.amazon.com/intel-Haswell-420...B015OOEQEC

$249 for 4GB RAM + 128GB SSD so it's a bit of premium over AliExpress. But if you're not comfortable dealing with vendors in China directly, it's worth the extra $15 for the peace of mind.

Last edited by Mossy (21-12-2015 08:24)

Re: Pianoteq benchmarks -- performance across various hardware

00:03.0 Audio device: Intel Corporation Haswell-ULT HD Audio Controller (rev 0b)
00:1b.0 Audio device: Intel Corporation Lynx Point-LP HD Audio Controller (rev 04)

Ok, thank you. Intel HDA seems to be just the familiy of Audio devices. The audiocodec for analog output can be found with aplay for example (here a Realtek ALC283 on my Bay Trail netbook):

 $ lspci | grep Audio
00:1b.0 Audio device: Intel Corporation Atom Processor Z36xxx/Z37xxx Series High Definition Audio Controller (rev 0e)

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: ALC283 Analog [ALC283 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Re: Pianoteq benchmarks -- performance across various hardware

Looks like Realtek ALC662

**** List of PLAYBACK Hardware Devices ****
card 0: HDMI [HDA Intel HDMI], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 7: HDMI 1 [HDMI 1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: HDMI [HDA Intel HDMI], device 8: HDMI 2 [HDMI 2]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: PCH [HDA Intel PCH], device 0: ALC662 rev1 Analog [ALC662 rev1 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

Re: Pianoteq benchmarks -- performance across various hardware

Mossy wrote:

Looks like Realtek ALC662

Yes, up to 24bit/96kHz, nice option to optimize the latency (when CPU-power is no limitation). Thanks.

Re: Pianoteq benchmarks -- performance across various hardware

Just a small note to say that my Asus X453M laptop ( CPU Baytrail N2840 ) gives a performance index of about 18, pretty much in line with your findings, which were very helpful in deciding to purchase Pianooteq ( and not race out and get a new laptop as well ! ).  I also tried the demo, of course, but your information was useful too.

A minor point about your measurements is that what seems to mater is the number of real floating point units.  The number of cores may not be as significant.  I note the behaviour of some AMD chips which have four integer units, but only two FP units.  Just a thought.

StephenG

Re: Pianoteq benchmarks -- performance across various hardware

I just built a Ryzen 1600X (6c/12t, 3.6/4.0GHz base/turbo) system, and I saw Performance Index in Pianoteq 6 as high as 125 or so.

Right now with a polyphony of 60, and the CPU set to 2.2GHz, PI is 95.


I tried benchmarking Pianoteq 6 on my old FX-8350 and this 1600X, same version, same 11 minute MIDI file:

time Pianoteq\ 6 --preset "Grotrian Concert Royal" --bit-depth 32 --rate 48000 --midi test.midi --wav output.wav

FX-8350 (stock, 4GHz)
real 1m39.393s
user 3m11.149s
sys 0m1.277s

1600X (stock, 3.6GHz)
real 0m57.783s
user 1m48.699s
sys 0m0.429s

So the 1600X is about 76% faster than the 8350. In both instances, Pianoteq 6 only maxed out two threads on each CPU. So, the 1600X could run more instances than the 8350 if I needed to.

Re: Pianoteq benchmarks -- performance across various hardware

Hello Mossy,

i am new to Pianoteq. I just downloaded the demo and checked it out.
I am still missing a special midi / audio interface.
I today called a local musicshop and he recommended the Steinberg ur22 to me, which is around 125€.
I am normally using my onboard sound for windows/games and checked pianoteq 6 with that.
I was amazed that I could use it with DirectSound with 4ms latency buffer without ASIO driver.
I installed ASIO4ALL and could lower the latency to 64 samples 1.3 ms (48KHz).
I could even cheat and set the sampling frequency to 96KHz and leave the buffer at 64 samples and still not dropouts with the ASIO4ALL.
I looked up the onboard sound chip and it is a Realtek ACL 1150 http://www.realtek.com.tw/products/prod...ProdID=328
It seems to be an amazing chip (Motherboard was expensive enough though ).
It seems i do not need an additional audio card.

Then I made a stress test. I went to the performance tab. Then i activated playing pianteq by computer keyboard.
I set maximum polyphony to Auto(optimistic).
Then I held the <tab> key, which is pedal and used the mouse to torture the virtual keyboard maximal.
I tried to use all key for around 5 minutes.
Polyphony was >250 almost all the time and performance index rose to 131.
No dropouts. For comparison i set the buffer size to 256 and 512 but there was no difference in sound during the torture test.

My computer is Big Tower with a Intel i7-4790k @4.0 GHz with 4.4 GHz turbo with 4 cores + HT.
It seems that baby will run the piano simulation very fine under any circumstance.

I am wondering why your 4790k did not had the same PI ?

BR
Sven

Re: Pianoteq benchmarks -- performance across various hardware

I made another check. I set sample rate to 192kHz without changing sample buffer (64 samples) means 0.3 ms latency and still no dropouts.
Obviosly the onboard soundchip has reserves.

Re: Pianoteq benchmarks -- performance across various hardware

All I know is stay away from Celeron.  Pianoteq hardly works at all.   Pianos don't play, but harpsichord is marginal.

Last edited by GRB (24-10-2017 15:19)
Pianoteq Pro 7.x - Kubuntu Linux 19.10 - Plasma Desktop - Hamburg Steinway

Re: Pianoteq benchmarks -- performance across various hardware

Do you have the midi file for the test?
I own a Raspberry pi 3... I would like to test it.

Pianoteq Pro - Bechstein - Blüthner - Grotrian - K2 - Kremsegg 1 & 2 - Petrof - Steingraeber - Steinway B & D - YC5
Kawai CL35 & MP11

Re: Pianoteq benchmarks -- performance across various hardware

GRB wrote:

All I know is stay away from Celeron.  Pianoteq hardly works at all.   Pianos don't play, but harpsichord is marginal.

I've used Pianoteq 5 for months with pianos and harpsichord on a Celeron N2840 based laptop.  It was fine.  Performance index was about 19 - nothing to get excited about, but perfectly usable and with no obvious (to me) sound degradation.

Be careful with these brand names - there are a huge number of Celeron CPUs all with quite different performance characteristics.

StephenG