A few weeks ago I wrote about “MySQL Performance at High Concurrency” and why it is important, which was followed up by Vadim’s post on ThreadPool in Percona Server providing some great illustration on the topic. This time I want to target an opposite question: why MySQL performance at low concurrency is important for you.
I decided to write about this topic as a number of recent blog posts and articles look at MySQL performance starting with certain concurrency as the low point. For example, MySQL 5.6 DBA and Developer Guide has a number of graphs starting with 32 connections at the low point. Mark Callaghan also starts with a concurrency of 8 in some of the results he publishes. All of this may make it look as though single-thread performance is no more a concern. I believe it still is!
First, performance at concurrency of 1 serves as an important baseline that offers the best response times (MySQL server does not use multiple threads to execute query in parallel at this point). Response times at the variety of concurrency levels can be compared with this baseline to see how they are affected and how the system scales with increasing concurrency in effect.
Second, there are many cases in which single-thread execution is what is going to happen — many batch jobs are written to be single-threaded. MySQL replication is single-thread too, and MySQL 5.6 brings some abilities of parallel replication but these apply only to some use cases. Many database maintenance operations such as “alter table” are run in single-thread too.
In fact, chances are your system actually runs at low concurrency most of the time. The majority of systems I see in the wild have Threads_running on average of 5 or less, at least the system is performing well, pointing to rather low concurrency in the normal system operation. Much higher concurrency in most cases is seen when the system is overloaded and suffering.
So what is the problem with looking at performance at just high-concurrency values? This might mislead you on how your production will be impacted. Think about, for example, a system that is slower at low concurrency but a lot faster at high concurrency (which is the typical situation). If you just look at performance at high concurrency you might not understand why the system is showing worse response times when you are actually running it in real life with low concurrency.
When it comes to benchmarks I would love to see results published starting from concurrency of 1 if possible so we can see the full picture. In terms of benchmarks, Mark Callaghan published an example where 8 sysbench tables are used which means the lowest concurrency you can run with to compare apples to apples is 8. Running a comparable benchmark with concurrency of 1 and the same tool is not possible.
Want to talk more about performance? Come to Percona Live MySQL Conference and Expo April 22-25 in Santa Clara, California, where “performance” is going to be a big topic 🙂
You are implying a lot from the fact that I haven’t published results for 1 to 7 threads.
Percona should publish more results on performance for less than 8 threads. I don’t think anybody disagrees with your point that low concurrency performance matters but you have not published enough results to match the conclusions you are making.
Mark,
I’m no sure what exactly do you disagree with ? My point is you publish with 8 threads and multiple of them because you are running sysbench with 8 tables and certain number of threads per table if I understand you correctly.
The point I’m making is what performance with low concurrency does matter and I’d like to encourage people not to start measuring performance at higher concurrency. Which of these do you disagree with ?
> Mark Callaghan also starts with a concurrency of 8 in some of the results he publishes.
> All of this may make it look as though single-thread performance is no more a concern. I believe it still is!
I don’t want to be the motivation for this post. But my primary point is that you haven’t published much in the way of single-threaded benchmarks. I think you should do more of that before writing a post like this.
> Mark Callaghan published an example where 8 sysbench tables are used which means the
> lowest concurrency you can run with to compare apples to apples is 8. Running a comparable
> benchmark with concurrency of 1 and the same tool is not possible.
It is possible with my framework. I just haven’t done it.
Mark,
You got me. You’re not the motivation but I just did not want to single out Oracle materials linked in the post first which start from 32 threads in many cases… I read your blog so you came as another example on my memory.
I personally always try to provide 1 thread baseline for the reasons mentioned, at least in case of performance comparison among with results for higher numbers.
My mistake about assuming it is not possible in your framework.
Peter,
stay tuned, I’m digging it too 😉
seems we’ll have a lot of materials to discuss during the conference..
Rgds,
-Dimitri
I am going to be posting more SSB results which are low concurrency, as well as ones at higher concurrency too, but mysql currently has a bug in which concurrent queries with joins, especially those that examine a lot of the same data degrade very quickly under concurrency. http://bugs.mysql.com/bug.php?id=68079 (Dimitri is working on it 🙂
Some results at http://mysqlha.blogspot.com/2013/03/mysql-56-single-threaded-read-only.html. All of the conclusions assume a benchmark with fast (100 to 200 usecs) queries:
* performance_schema overhead is too high for benchmarks
* innodb_checksum_algorithm=CRC32 is a big win
* 5.6 was ~9% slower for a cached database whether or not the PS was disabled
Opened a bug – http://bugs.mysql.com/bug.php?id=68825
Update-only workloads also show a regression — http://mysqlha.blogspot.com/2013/04/mysql-56-single-thread-update-only.html