December 29, 2008

XtraDB/InnoDB CPU bound benchmarks on 24cores server

Posted by Vadim |

One of our customers gave me a chance to run some benchmarks on 24-core (intel cpu based) server, and I could not miss it and ran few CPU-bound tasks there.

The goal of benchmarks was investigation of InnoDB-plugin and XtraDB scalability in CPU-bound load.

CPU specification:

PLAIN TEXT
CODE:
  1. processor       : 23
  2. vendor_id       : GenuineIntel
  3. cpu family      : 6
  4. model           : 29
  5. model name      : Intel(R) Xeon(R) CPU           E7450  @ 2.40GHz
  6. stepping        : 1
  7. cpu MHz         : 2394.011
  8. cache size      : 12288 KB
  9. physical id     : 3
  10. siblings        : 6
  11. core id         : 5
  12. cpu cores       : 6
  13. fpu             : yes
  14. fpu_exception   : yes
  15. cpuid level     : 11
  16. wp              : yes
  17. flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm
  18. bogomips        : 4788.59
  19. clflush size    : 64
  20. cache_alignment : 64
  21. address sizes   : 40 bits physical, 48 bits virtual
  22. power management:

I tested MySQL-5.1.30 with InnoDB plugin, Xtradb-1.0.2-1, and XtraDB-1.0.2-2 (rel2). XtraDB-rel2 has not been released yet, we still are doing internal QA for, but it will be ready soon. Main difference XtraDB-rel2 it contains additional scalability fixes for buffer_pool (split_buffer_pool_mutex patch).

my.cnf is

PLAIN TEXT
CODE:
  1. [mysqld]
  2. user=root
  3.  
  4. binlog_cache_size=1M
  5. default_table_type=MYISAM
  6. ft_min_word_len=4
  7.  
  8. innodb_additional_mem_pool_size=16M
  9. innodb_buffer_pool_size=15G
  10. innodb_data_file_path=ibdata1:10M:autoextend
  11. innodb_file_per_table=1
  12. innodb_flush_log_at_trx_commit=2
  13. innodb_log_buffer_size=8M
  14. innodb_log_files_in_group=2
  15. innodb_log_file_size=512M
  16. innodb_status_file=0
  17. innodb_thread_concurrency=0
  18.  
  19. innodb_io_capacity=1000
  20. innodb_write_io_threads = 16
  21. innodb_read_io_threads = 16
  22.  
  23.  
  24. join_buffer_size=1M
  25. max_allowed_packet=1M
  26. max_connections=3000
  27. max_heap_table_size=64M
  28. max_prepared_stmt_count=1000000
  29. query_cache_size=0
  30. skip_grant_tables
  31. skip_locking
  32. sort_buffer_size=64K
  33. table_cache=2048
  34. thread_cache_size=16
  35. thread_concurrency=16
  36. thread_stack=196K
  37. tmp_table_size=64M
  38. transaction_isolation=REPEATABLE-READ
  39. local-infile=1

At first I tried sysbench oltp read-only with 10mil rows (the datasize is about 2.5GB), uniform distribution.
The results you can see there: (Results are removed for additional checking)

In the next run I tested sysbench oltp read-write load, and the results are:

Here starting 16 threads the result is dropping down with the same speed as it grew, and with 128 connections we have the same TPS as with 1 connection (and it is on 24-cores box!). XtraDB is slightly better there than InnoDB, but nothing special to be proud of. We definitely we will look how to fix it as next step, read detailed investigation what is the reason of performance drop in next post.

And last one workload I tried is TPCC-like benchmark (you can get it on https://launchpad.net/perconatools), with 100 Warehouses (about 9.5GB datasize).
The result is:

Here the result grows up to 16 connections, but after that InnoDB-plugin is dropping down. XtraDB and XtraDB-rel2 seem quite better, I guess this is mostly because fixes to rw_locks and to buffer_pool mutex (in rel2).

Conclusion: As read-only workload seems fine, read-write cases is something to worry about in 16+ cores boxes.
Intel Based 24 Core Servers are not mainstream these days but as number of cores is increasing now at the same pace as CPU frequency before we believe they are very soon to come. Also in real production there may be not a lot database fits "in-memory" cases - but on other hand 64-128GB RAM per box is not something rare already and recommendation to fit at least active dataset in memory is one we use for our customers.

Posted by Vadim @ 12:54 pm :: Innodb, benchmarks, xtradb  
Print This Post Print This Post book mark XtraDB/InnoDB CPU bound benchmarks on 24cores server in del.icio.usdel.icio.us :: submit XtraDB/InnoDB CPU bound benchmarks on 24cores server to digg.comdigg  
Comment RSS  
Related posts: :Fix of InnoDB/XtraDB scalability of rollback segment::Looking on 5.4 – IO bound benchmarks::5.4 in-memory tpcc-like load:
 

6 Comments »

  1. 1. dim

    Hi Vadim,

    did you try to replay the same tests with innodb_thread_concurrency=16 ?

    Rgds,
    -Dim

    Comment :: December 30, 2008 @ 6:04 am

  2. 2. Vadim

    Dim,

    Nope I did not.
    I know it may fix scalability a bit, but innodb_thread_concurrency=16 on 24-cores box does not make sense.

    Comment :: December 30, 2008 @ 9:57 am

  3. 3. peter

    Interesting point though. This is how things degrade with unrestricted concurrency… Often you can get better performance restricting Innodb Thread Concurrency… but our goal is rather fixing internal scaling limits so it is not required.

    Comment :: December 30, 2008 @ 10:36 am

  4. 4. Torsten Brumm

    Hi Vadim, did you like to do this test on a 64 Core System with a lot of RAM again ;-) ?

    Comment :: February 19, 2009 @ 5:22 am

  5. 5. Vadim

    Torsten,

    That would be interesting !

    Comment :: February 19, 2009 @ 11:41 am

  6. 6. Torsten Brumm

    @Vadim,
    pls mail me directly your needs and i will sent you details about login ;-)

    Would be interessting to see your results, we only use this box (we have two) for 10% max….never saw a higher load on the box….

    Comment :: February 20, 2009 @ 1:07 am

 

Subscribe without commenting

Trackbacks/Pingbacks