Thanks

thanks for pointing at the obvious, it was missed very much. I’m surprised noone else pointed out the scaling of the graphs x-axis is not OK.

actually, from my POV what we see is that
- scalability is still far from linear
- there must be still some evil issues lurking, the performance hit from adding more threads should not bring 16-core performance down to the performance of 2 threads (== 2 cores)

luckily oracle is licensed per socket, otherwise one would be able to plot in the point (number of threads) where the hardware cost for “Scaling” breaks even with an oracle license.
I hope you understand my point considering that even midrange servers can hit 48cores today.

If I look at it from the other end, I wonder what hardware would be needed to scale innodb up to the performace of a 128core unix iron + better scaling database.

anyway, I think such scalabilty benchmarks are highly neccessary and will help (continued) improvement a lot, removing locks here and there along the way

I put “scale well” between quotes because the scale of your graph is misleading. The scaling looks linear (and even better than linear) but is not because X-axis is logarithmic. In fact, if you draw the same graph with a linear X-axis you will see that it doesn’t really scale well. The improvement comes from a better usage of your CPU.

# Runs sysbench increasing thread count each time to 512 threads
# http://www.mysqlperformanceblog.com/2010/02/28/mysql-5-5-m2-scalability/

sysbench –test=oltp –oltp-table-size=10000000 –mysql-user=sbtest –mysql-password=yourpasswd \
–mysql-socket=/var/lib/mysql/mysql.sock –init-rng=1 –num-threads=1 –max-requests=0 \
–oltp-dist-type=uniform –max-time=180 prepare

for i in 1 2 4 8 16 32 64 128 256 512
do
sysbench –test=oltp –oltp-table-size=10000000 –mysql-user=sbtest –mysql-password=yourpasswd \
–mysql-socket=/var/lib/mysql/mysql.sock –init-rng=1 –num-threads=$i –max-requests=0 \
–oltp-dist-type=uniform –max-time=180 run
done

I would be careful with scalability claims. You’ve done this benchmark and you have results for this benchmark. Things has gotten a lot better and it is great. However it does not mean all workloads will scale as well too.

Workloads expose different bottlenecks which can affect scalability differently.

Also I think there are 2 different aspects – one is showing improvement with multiple threads and cores and not failing down too much with many connections, another is scalability factor.

In your benchmark we see performance with 16 threads which is the peak is 7 times better than with one thread. Assuming single thread uses a bit over 1 core (most of the work is done by connection thread) we get up to double potential capacity.

It is hard to judge without CPU usage data but I suspect there is some CPU which is free (or wasted doing spinlocks etc due to scalability limitations)

Still, benchmarks against 5.5 would be good.

Also, many people are confused about xtradb vs. percona performance build of mysql. (heck, I am a bit, I know they’re different in many 5.0 versions but I’m not sure if they merge at some point).

I remember seeing similar slides in a presentation by Giuseppe – iirc 5.4 was outperforming XtraDB in a read-only sysbench. I believe Giuseppe took his slides from one of Dimitri’s benchmarks that was posted about the same time. In which case, a more up to date post shows how the numbers have changed (spoiler, XtraDB wins):

http://dimitrik.free.fr/blog/archives/2009/12/entry_101.html