MySQL Performance Blog » benchmarks

How many fsync / sec FusionIO can handle

Vadim — Thu, 04 Mar 2010 16:09:44 +0000

I recently was asked how many fsync / sec ( and therefore durable transactions / sec) we can get on FusionIO card.

It should be easy to test, let's take sysbench fileio benchmark and run, the next command should make it:

./sysbench --test=fileio --file-num=1 --file-total-size=50G --file-fsync-all=on --file-test-mode=seqrewr --max-time=100 --file-block-size=4096 --max-requests=0 run

PLAIN TEXT

CODE:

Operations performed: 0 Read, 922938 Write, 922938 Other = 1845876 Total
Read 0b Written 3.5207Gb Total transferred 3.5207Gb (36.052Mb/sec)
9229.35 Requests/sec executed

So that's 9229.35 req/sec, which is pretty impressive.

For comparison the same run on PERC 6i RAID10 with BBU:

PLAIN TEXT

CODE:

Operations performed: 0 Read, 4832661 Write, 4832661 Other = 9665322 Total
Read 0b Written 18.435Gb Total transferred 18.435Gb (20.975Mb/sec)
5369.62 Requests/sec executed

which gives us 5369.62 req/sec.

Note this is for single thread, and in MySQL/InnoDB multi-thread load you may get more transactions per second with group commit ( which is back to live in InnoDB-plugin / XtraDB )

Entry posted by Vadim | 11 comments

Add to: | | | |

Maximal write througput in MySQL

Vadim — Mon, 01 Mar 2010 06:53:08 +0000

I recently was asked what maximal amount transactions per second we can get using MySQL and XtraDB / InnoDB storage engine if we have high-end server. Good questions, though not easy to answer, as it depends on:

- durability setting ( innodb_flush_log_at_trx_commit = 0 or 1 ) ?
- do we use binary logs ( I used ROW based replication for 5.1)
- do we have sync_binlog options.

So why would not take these as variable parameters and run simple benchmark.
I took sysbench update_key scenario ( update indexed field on simple table)
and used Dell PowerEdge R900 with 16 cores, FusionIO as storage for table and RAID 10 with BBU as storage for innodb log files, innodb system table space and binary logs. And I used Percon-XtraDB-5.1.43-9.1 for benchmarks. All used partitions are formatted in XFS and mounted with nobarrier option.

I run update key for various threads and with next parameters

trx_commit=0 : innodb_flush_log_at_trx_commit = 0 and no binary logs
trx_commit=1 : innodb_flush_log_at_trx_commit = 1 and no binary logs
trx_commit=0 & binlog : innodb_flush_log_at_trx_commit = 0 and binary logs
trx_commit=1 & binlog : innodb_flush_log_at_trx_commit = 1 and binary logs
trx_commit=1 & binlog & sync_bin : innodb_flush_log_at_trx_commit = 1 and binary logs and sync_binlog=1

There are results I get:

I found results being quite interesting.
with innodb_flush_log_at_trx_commit = 0 maximal tps is 36332.02 tps, which drops to 23115.04 tps as
we switch to innodb_flush_log_at_trx_commit = 1. As we use RAID10 with BBU, I did not expect the drops is going to be significant. In second case InnoDB spends

With enabling binary logs, the results drops to 17451.01 tps with innodb_flush_log_at_trx_commit = 0 and to 12097.39 tps with innodb_flush_log_at_trx_commit = 1. So with binary logs serialization is getting even worse.

Enabling sync_binlog makes things really bad, and maximal results I have is
3086.7 tps. So this is good decision if binary log protection is worth such drop.

UPDATE ( 3/4/2010 )

Results with innodb_flush_log_at_trx_commit = 2

Results with innodb_flush_log_at_trx_commit = 2 and binlogs

Entry posted by Vadim | 7 comments

Add to: | | | |

READ-COMMITED vs REPETABLE-READ in tpcc-like load

Vadim — Thu, 11 Feb 2010 21:27:54 +0000

Question what is better isolation level is poping up again and again. Recently it was discussed in InnoDB : Any real performance improvement when using READ COMMITED isolation level ? and in Repeatable read versus read committed for InnoDB .
Serge in his post explains why READ COMMITED is better for TPCC load, so
why don't we take tpcc-mysql bencmark and check on results.

I took 3 different datasets 1000w (100GB of data), 300w (30GB) and 10w (1GB) for box with 32GB of RAM and buffer_pool 26GB. Latest case 10w is interesting as I expect a lot of contention on row level having small dataset.
I used as usually tpcc-mysql benchmark with 16 and 32 (for 10w) concurrent users.

Also I had binary log enabled on RBR mode (as READ-COMMITED does not support STATEMENT based replication in 5.1), and everything was run under XtraDB-9 ( based on InnoDB-plugin 1.0.6)

So there are results:

For 1000w:

There READ-COMMITED seems more preferable, however difference is very small.

For 300w:

I would say both modes are even there, there is no winner for me.

for 10w:

Almost indentical resutls in this case.

However, however.
For 10w run I got 60 Deadlock errors "1213, Deadlock found when trying to get lock; try restarting transaction" in READ-COMMITED mode, and 0 Deadlocks in REPEATABLE-READ mode.

I understand that 60 deadlocks for total 2704687 transactions can be ignored, but it seems you have better chance to get DEADLOCK in READ-COMMITED then in REPEATABLE-READ.

So both modes looks even for me, though some facts to consider:

READ-COMMITED is used rare than REPEATABLE-READ (default), that is less-tested
READ-COMMITED does not work with statement-based replication in 5.1
with READ-COMMITED you may have more DEADLOCKS

Entry posted by Vadim | 5 comments

Add to: | | | |

Introducing tpce-like workload for MySQL

Vadim — Mon, 08 Feb 2010 17:30:44 +0000

We have been using tpcc-mysql benchmark for long time, and there many results published in our blog, but that's just single workload. That's why we are looking into different benchmarks, and one
of them is TPCE. Yasufumi made some efforts to make TPCE working with MySQL, and we are making it available for public consideration.

You can download it from our Lauchpad Percona-tools project, it's
bzr branch lp:~percona-dev/perconatools/tpcemysql

Important DISCLAIMER:
Using this package you should agree with TPC-E License Agreement,
which in human words is:

You can't name results as "TPC Benchmark Results"
You can't compare results with results published on http://www.tpc.org/ and you can't pretend the results are compatible with published by TPC.

And we are not going to do anything from that, your primary goals is XtraDB/InnoDB performance research and/or compare with available Storage Engines for MySQL.

The workload in tpce is quite different from tpcc. Tpcc is write intensive, while tpce
is read oriented.
To give more details, there is stats for 10 seconds:

PLAIN TEXT

CODE:

| Com_select | 46272 |
| Com_update | 5214 |
| Com_delete | 385 |
| Com_insert | 3468 |
| Com_commit | 5404 |

The result is quite chatty,

PLAIN TEXT

CODE:

| | [MEE] | [DM] | [CE] |
sec. | TR, MF | DM | BV, CP, MW, SD, TL, TO, TS, TU | MEEThreads, ReqQueue
(1st line: count, 2nd line: 90%ile response [msec.])
260 | 402, 39, 0, 195, 532, 749, 588, 342, 415, 816, 88 | 30, 0
20, 60, 0, 30, 20, 20, 20, 50, 20, 310, 60
270 | 395, 40, 0, 201, 608, 842, 608, 358, 449, 833, 89 | 30, 0
30, 40, 0, 30, 20, 20, 20, 50, 20, 300, 50

but it allows you to see count of 11 different transactions per 10 secs and 90% response time.

and final result

PLAIN TEXT

CODE:

[TradeResult(TR) transaction]
Succeed: 150243
Lated: 0
Retried: 3
Failed: 0
41.7342 TpsE

where you can see count of successful TR (TradeResult) transactions, and
the summary result in TpsE (transactions per seconds).

Expect our results soon!

Entry posted by Vadim | No comment

Add to: | | | |

New OLAP Wikistat benchmark: Introduction and call for feedbacks

Vadim — Fri, 29 Jan 2010 03:08:47 +0000

I've seen my posts on Ontime Air traffic and Star Schema Benchmark got a lot of interest
(links:

).
However benchmarks by itself did not cover all cases I would want, so I was thinking about better scenario. The biggest problem is to get real big enough dataset, and I thank to Bradley C. Kuszmaul, he pointed me on Wikipedia statistics on access to Wikipedia pages, and thank to Domas, who made stats accessible. Link to the archives: http://dammit.lt/wikistats/archive/ or the original Domas's announcement .

Although the table does not have very much different information, I think it is good enough to represent cases you can face in Web application ( log processing, page visits, clickstream, etc).

I made some efforts to normalize data to have model in classic star schema and prepared queries that could be run on proposed dataset (John Sichi, lead of LucidDB helped me to draft some queries).
You can see details on our Percona Wikistat benchmark Wiki.

I have next goals with proposed benchmark:

Compare engines in OLAP queries for planning, predicting growth, analyzing access patterns to wiki pages, draw trends.
Compare engines in statistical queries for end users, which can be executed in real-time. I.e. How many times that or another page was accessed yesterday vs today.
Understand specific features and characteristic of each engine.
Compare throughput on simple queries (queries and scenario to be drafted yet)
Check ability to load data and serve queries at the same time ( availability during data load ) (queries and scenario to be drafted yet)

So in proposed schema I have four tables:
pagestat (fact table), and pages, datesinfo, projects (dimensions tables).

Dimensions tables are supposed to be static and not changed, and we can change datasize
by varying amount of months loaded into fact table (so this is scale factor).

EER diagram

( made with MySQL Workbench )

In current dataset, which you can download from Amazon snapshot (name: “percona-wikistat”, ID:snap-a5f9bacc) we have:

Table pages: 724.550.811 rows. data size: 40476M
Table datesinfo: 9624 rows, one entry represents 1 hour
Table projects: 2025 rows
Table pagestats
Data for 2009-06: # 3.453.013.109 rows / size 68352M
Data for 2009-07: # 3.442.375.618 rows / size 68152M

So with two months of stats we have about 172GB of data with about 7 billion rows in fact table.

Example of query ( again, full list on Benchmark Wiki)

PLAIN TEXT

SQL:

SELECT project, sum(page_count) sm
FROM pagestat
JOIN datesinfo di ON ( di.id=date_id )
JOIN projects p ON (p.id=project_id )
WHERE di.calmonth=7 AND di.calyear=2009
GROUP BY project
ORDER BY sm DESC
LIMIT 20;

I am going to load data and run queries against available engines:

MySQL MyISAM / InnoDB (to have reference results)
InfoBright
InfiniDB
MonetDB
LucidDB
Greenplum

and I will report my results ( so stay with MySQLPerformanceBlog )

I'd like also to test also Paraccel, Vertica and KickFire systems, but I do not have access to.

I welcome your feedback on the benchmark, and what else you would like to see here.

Entry posted by Vadim | 16 comments

Add to: | | | |

Fast storage: 8 SSD Intel X-25M 80GB benchmarks

Vadim — Mon, 18 Jan 2010 16:33:01 +0000

I appreciate opportunity Jos van Dongen from Tholis Consulting gave me. He granted me access to servers with 8 attached Intel X-25M 80GB MLC cards. The cards attached to 2 Adaptec 5805 raid controllers, with 4 cards per controller.

The cost of setup is 8 x 260$ (X-25M) + 2x500$ (Adaptec 5805) = ~3000$.
Available space varies in depends on raid setup from 300GB to 600GB.

The logical comparison is to compare results with FusionIO 320GB MLC card, so I will copy results from FusionIO 320GB MLC benchmarks.

For benchmarks I used sysbench fileio benchmark.
All raw results are available on Percona benchmarks wiki, there I will highlight most interesting points.

Couple words on tested setups. We used two configurations:

In first runs (software) each card connected as individual card to Adaptec, so in OS we see 8 individual cards. Card configured in software RAIDs: RAID0, RAID10, RAID5, RAID50
In second round (marked as hardware) each Adaptec is configured as hardware RAID0 over 4 cards, so in OS we see 2 devices. Devices connected in software RAIDs: RAID0 and RAID10 with different OS schedules over each device.

I should highlight I do not see usage in production for RAID0, as in my opinion SSD cards
are not reached enough level of reliability yet (see comments to post FusionIO 320GB MLC benchmarks), however I put results here to show theoretical maximal results.

So let's start with random reads:

I'd say SSDs show comparable results with FusionIO on 16+ threads, however on 4-8 threads difference is significant. On SSD you can get 160MB/sec for 4 threads and 260MB/sec for 8 threads.

Random writes:

There couple things to note (beside SSD is doing much worse than FusionIO).
1. Something is wrong with scaling random writes in this setup. It is point for research,
I think there some serialization in Linux software raid or Linux scheduler or on Adaptec hardware level.
2. Cards connected in hardware raids show worse results than card connected as single devices (you can see results on http://www.percona.com/docs/wiki/benchmark:ssd:start, in summary table, randwr rows)
3. For cards connected in hardware raids, DEADLINE performs much worse than CFQ.

Sequential reads:

- For sequential reads you can get pretty decent results from SSD (230MB/sec on 4+ threads)
- Cards connected in hardware raid is doing much better
- DEADLINE outperforms CFQ here (which is different from random writes)
- Software raid0 performed pretty bad, so I chosen to show hardware-raid0 results

Sequential writes:

I'd say sequential writes is hard task for both SSDs and FusionIO, it does not scale well.
You may want to look into another options if your load requires sequential writes (e.g. like I
put InnoDB transactional logs on rotation based drives instead of SSD in my InnoDB on FusionIO benchmarks).

So in summary I can say

With SSD drives you can get decent results for random and sequential reads.
I think it is competitive vs FusionIO if we look into price/performance analysis (remember FusionIO is twice as expensive)
Random writes did not work for me as expected, and this is point for investigations
RAID5, as expected, is only competitive for reads, but not writes
Complexity of having 8 SSD drives may be significant. You may want to
look into different options: connect them in software or hardware raids, what scheduler to
pick, etc. I suggest to run sysbench fileio or similar (i.e. iozone) to check if you get performance you expect
In my opinion maintaining 8 SSD cards per server is much more hassle than
deal with single FusionIO card, however there is important point that with SSD you may
"hot"remove and insert cards, while for FusionIO, which put into PCI-E bus, you need
to shutdown server to replace it

Entry posted by Vadim | 13 comments

Add to: | | | |

InnoDB, InnoDB-plugin vs XtraDB on fast storage

Vadim — Wed, 13 Jan 2010 08:52:38 +0000

To continue fun with FusionIO cards, I wanted to check how MySQL / InnoDB performs here. For benchmark I took MySQL 5.1.42 with built-in InnoDB, InnoDB-plugin 1.0.6, and XtraDB 1.0.6-9 ( InnoDB with Percona patches).
As benchmark engine I used tpcc-mysql with 1000 warehouses ( which gives around 90GB of data + indexes) on my workhourse Dell PowerEdge R900 ( details about box ).

On storage configuration: FusionIO 160GB SLC and 320GB MLC cards are configured in software RAID0 to store InnoDB datafiles. For InnoDB logs and system tablespace I used partition on regular RAID10 with regular hard drives, here I followed Yoshinori Matsunobu's recommendations http://yoshinorimatsunobu.blogspot.com/2009/05/tables-on-ssd-redobinlogsystem.html and taking fact that FusionIO is not perfect for sequential writes http://www.mysqlperformanceblog.com/2010/01/11/fusionio-320gb-mlc-benchmarks/

Full results I put on page http://www.percona.com/docs/wiki/benchmark:fusionio:mysql:start, here are my thoughts and interesting facts.

First, chart with results for InnoDB vs InnoDB-plugin during runs (values are in new order transactions per minute, more is better) :

As you see InnoDB-plugin is doing much better here, it allows to utilize multiple IO threads,
which as we saw ( http://www.mysqlperformanceblog.com/2010/01/11/fusionio-320gb-mlc-benchmarks/ ) is necessary to get most throughput from FusionIO.

Also you may see from graph some waves for InnoDB-plugin. Here we observe innodb_adaptive_flushing in action (which is ON by default), and I think innodb_adaptive_flushing in InnoDB-plugin is not quite balanced, it may do overaggressive flushing, when it is not necessary.

But looking on CPU stats (see graph later), I guess InnoDB-plugin spends most time in buffer_pool mutex, contention here is not fully resolved yet in InnoDB-plugin.

Now, let's take XtraDB. In additional to multiple IO threads, we have patch to decrease contention on buffer_pool mutex, plus separate purge thread. Also we use different adaptive_checkpoint algorithm.

The results are:

So I guess buffer_pool improvements play here for XtraDB, and looking on summary result:

InnoDB 9439.316 NOTPM
InnoDB-plugin-1.0.6 15299.333 NOTPM
XtraDB-1.0.6-9 26160.551 NOTPM

InnoDB-plugin is 1.6x times better InnoDB, and XtraDB is 1.7x times better InnoDB-plugin.

Now on CPU usage and disk utilization.

Disk throughput:

CPU (user) usage:

Even with improvements, XtraDB performs less 150MB/s in disk writes (from benchmarks we
saw FusionIO can do much more) and with 45-50% of idle CPU.

I assume we still see significant contentions inside XtraDB, and there big room for improvements. As for InnoDB-plugin, I'd wish InnoDB team makes some actions on buffer_pool mutex problem.

Finally I wanted to check what if we put innodb transactional logs and system tablespace on FusionIO also, there is graph for that:

It is not so bad, with final result 23038.283 NOTPM, it is only about 12% worse than with logs on separate partition.

And to make reference point, I run the same but with all files on RAID10 with regular disks,
the graph is there:

with final result: 2873.783 NOTPM ( about 88% worse than all files on FusionIO)

To summarize

MySQL InnoDB/InnoDB-plugin/XtraDB is not fully able to utilize throughput of FusionIO. XtraDB is doing better job with internal contention, but much more can be done.
Still you can have very impressive performance improvement in IO-intensive or IO-bound workloads. You may want to use InnoDB-plugin or XtraDB to get better results.
Putting logs on separate partition may be good idea, but only if you have possibility to do that. Making special setup for that may be not worth improvements

Entry posted by Vadim | 10 comments

Add to: | | | |

FusionIO 320GB MLC benchmarks

Vadim — Tue, 12 Jan 2010 04:15:22 +0000

After my previous benchmarks of FusionIO 160GB SLC card, FusionIO team sent me for the tests another card, FusionIO 320GB MLC. I should say I really appreciate an opportunity to play with this card and with combination of two cards.

This card is also not cheap, the price I can find on dell.com is $6,829.99, which is almost the same as for 160GB card, but with doubled space,
and which gives me 21$/GB for 100% formatted card, and 29$/GB for card with 75% of space allocated. Effect of 25% reserved space you will see in the results.

The full numbers are available on our Wiki benchmarks page, couple words about benchmarks.

I did sysbench fileio runs on 100% space allocated card and on 75% space allocated (240GB of
available disk space), and on 32GB and 100GB filesizes. After that I took two cards into software RAID0, and tried 100GB filesize in this configuration. Block size is 16KB in all cases.

The results for random write is here:
(Results are in MB/s (more is better) for random write test, pagesize 16KB )

threads	32GB filesize, 100% space formatted	32GB filesize, 75% space formatted	32GB filesize, SLC	100GB filesize, 75% space formatted	100GB filesize, RAID0 over 2 cards
1	83.917	148	134.33	149.33	132.91
4	130.83	418.88	443.59	411.02	430.45
8	177.34	475.17	566.54	455.25	692.58
16	161.74	488.64	554.73	472.3	784.31
32	150.9	491.41		457.6	718.81
64	146.08	486.97		387.17	740.21

or in chart form:

(if legend is unreadable, it is:

32GB filesize, 100% space formatted
32GB filesize, 75% space formatted
100GB filesize, 75% space formatted
100GB filesize, RAID0 over 2 cards

I hope Google Docs will fix the issue with font for the legend sometime
)

So there some numbers to consider:

With 25% reserved space you can get almost 3x throughput even with
small enough files
SLC card gives about 15-20% more in throughput
With 25% reserved space, I was able to get 700 MB/s in sequential reads ( 400 MB/s in random reads) and 500 MB/s for random writes.
95% response time in most cases are impressive, very often it is < 1ms (in hundreds microseconds range)
Sequential writes still behave strange, dropping throughput with increased concurrency
To get most from card you need multi-threaded IO (4-16 threads)
With RAID0 you can get doubled space with very interesting numbers, 750 MB/s in random reads and 800 MB/s in random writes ( and 1.2GB/s in sequential reads)

Now having these cards and numbers, I am going to benchmark MySQL with FusionIO as storage.

Entry posted by Vadim | 19 comments

Add to: | | | |

Star Schema Bechmark: InfoBright, InfiniDB and LucidDB

Vadim — Fri, 08 Jan 2010 05:51:20 +0000

In my previous rounds with DataWarehouse oriented engines I used single table without joins, and with small (as for DW) datasize (see http://www.mysqlperformanceblog.com/2009/10/02/analyzing-air-traffic-performance-with-infobright-and-monetdb/, http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/, http://www.mysqlperformanceblog.com/2009/11/02/air-traffic-queries-in-infinidb-early-alpha/). Addressing these issues, I took Star Schema Benchmark, which is TPC-H modification, and tried run queries against InfoBright, InfiniDB, LucidDB and MonetDB. I did not get results for MonetDB, will explain later why. Again primary goal for test was not to get just numbers, but understand specifics of each engine and their ability to handle amount of data and execute queries.

All details I have are available on our Wiki http://www.percona.com/docs/wiki/benchmark:ssb:start and the specification of benchmarks you can get there http://www.percona.com/docs/wiki/_media/benchmark:ssb:starschemab.pdf.

I generated data with scale factor = 1000, which gave me 610GB of data in raw format and loaded into each engine.

There difference in engines gets into play. While InfoBright and InfiniDB does not need indexes at all (you actually can't create indexes here), they needed for LucidDB and MonetDB, and it changes load time and datasize after load significantly. The numbers
I put in results do not include indexing time, but it also should be considered.

And indexes are exactly reason why I could not get results for MonetDB, there I faced issue
I was not prepared for. MonetDB requires that index fits into memory during indexing procedure, and for 610GB the index may get to 120GB size, and I did not have that amount of memory ( the box is only 16GB of RAM). MonetDB experts recommended me to extended
swap partition to 128GB, but my partitions layout was not really prepared for, I just did not expect I need big swap partition.

Loading
So load time.
InfiniDB can really utilize all available cores/cpus in systems ( I run benchmark on 8 cores box), and it allowed to load data faster than other engines. Though LucidDB and MonetDB
are also have multi-thread loaders, only InfoBright ICE used single core.

InfiniDB: 24 010 sec
MonetDB: 42 608 sec (without indexes)
InfoBright: 51 779 sec
LucidDB: 140 736 sec (without indexes)

I should note that time to create indexes in LucidDB was also significant and exceeded loading time. Full report on indexes are available here http://www.percona.com/docs/wiki/benchmark:ssb:luciddb:start

Data size

Size after load is also interesting factor. InfoBright is traditionally good with compression,
though compression rate is less than in case with AirTraffic table. I was told this is because
lineorder table comes not in sorted order, which one would expect in real life. Actually
the same complain I heard from InfiniDB experts - if put lineorder data in sorted order, loading
time can decrease significantly.

Datasize after load:
InfoBright: 112G
LucidDB: 120GB (without indexes)
InfiniDB: 626GB
MonetDB: 650GB (without indexes)

Queries time

Now on queries time.
Full results you can find on page http://www.percona.com/docs/wiki/benchmark:ssb:start,
and graph is below. There couple comments from me.

InfoBright was fully 1 CPU bound during all queries. I think the problem
that engine can use only single cpu/core is getting significant limitation
for them. For query 3.1 I got the surprising result, after 36h of work I got
error that query can't be resolved by InfoBright optimizer and I need
to enable MySQL optimizer.

InfiniDB is otherwise was IO-bound, and processed data fully utilizing
sequential reads and reading data with speed 120MB/s. I think it allowed
InfiniDB to get the best time in the most queries.

LucidDB on this stage is also can utilize only singe thread with results sometime better,
sometime worse than InfoBright.

Results:

Query	InfoBright	InfiniDB	LucidDB
Q1.1	48 min 21.67 sec (2901.67 sec)	24 min 26.05 sec (1466.05 sec)	3503.792 sec
Q1.2	44 min 55.37 sec (2695.37 sec)	24 min 25.83 sec (1465.83 sec)	2889.903 sec
Q1.3	45 min 53.49 sec (2753.49 sec)	24 min 27.25 sec (1467.25 sec)	2763.464 sec
Q2.1	1 hour 54 min 27.74 sec (6867.74)	19 min 44.35 sec (1184.35 sec)	9694.534 sec
Q2.2	1 hour 13 min 33.15 sec (4413.15)	19 min 49.56 sec (1189.56 sec)	9399.965 sec
Q2.3	1 hour 8 min 23.41 sec (4103.41)	19 min 52.27 sec (1192.25 sec)	8875.349 sec
Q3.1	NA	19 min 11.23 sec (1151.23 sec)	16376.93 sec
Q3.2	3 hours 30 min 17.64 sec (12617.64 sec)	19 min 28.55 sec (1168.55 sec)	5560.977 sec
Q3.3	2 hours 58 min 18.87 sec (10698.87 sec)	19 min 58.29 sec (1198.29 sec)	2517.621 sec
Q3.4	1 hour 41 min 41.29 sec (6101.29 sec)	12 min 57.96 sec (777.96 sec)	686.202 sec
Q4.1	8 hours 53 min 52.55 sec (32032.55 sec)	32 min 57.49 sec (1977.49 sec )	19843.213 sec
Q4.2	5 hours 38 min 7.60 sec / 5 hours 36 min 35.69 sec (20195.69 sec)	33 min 35.45 sec (2015.45 sec)	15292.648 sec
Q4.3	12 hours 58 min 4.27 sec (46684.27 sec)	33 min 47.32 sec (2027.32 sec)	7241.791 sec

Graph with results (time in sec, less time is better)

Conclusions

InfiniDB is doing just great using available CPU cores full IO bandwidth reading from disk. You can see more details on InfiniDB scalability on InfiniDB's blog http://infinidb.org/infinidb-blog/mysql-parallel-query-processing-of-ssb-queries-via-infinidb-.html
SSB benchmark may be not good for InfoBright, the synthetic nature of benchmark
does not allow InfoBright to show better results. But I hope InfoBright will be able to reuse multi-cores / multi-disks soon.
I'd like MonetDB is able to use disk to build indexes, not only rely on available memory
Taking complains on SSB I am looking to get another more realistic dataset and
compare bigger set of available DW solutions

Entry posted by Vadim | 37 comments

Add to: | | | |

Redis Benchmarks on FusionIO (Round 1)

Ryan Lowe — Thu, 10 Dec 2009 19:29:32 +0000

Peter took a look at Redis some time ago; and now, with the impending 1.2 release and a slew of new features, I thought it time to look again.

One of the more interesting features in 1.2 is the ability to operate in "append-only file persistence mode", meaning Redis has graduated from a semi-persistent to a fully-persistent system! Using the redis-benchmark script included, I ran the following command

./redis-benchmark

in five modes:

1 - In-Memory
I set "save 900000000 900000000" so nothing would be written to disk during the tests.
2 - Semi-Persistent
I set "save 1 1" so that changes would be flushed to disk every second (assuming there was at least one change the previous second).
3 - Fully Persistent
I set appendonly yes and appendfsync always, which calls an fsync for every operation.
4 - Semi-Persistent
I set appendonly yes and appendfsync no, which is about as persistent as MyISAM. An fsync is never explicitly called, rather we're relying on the OS.
5 - Semi-Persistent
I set appendonly yes and appendfsync everysec, which explicitly calls an fsync every second.

(By default, redis-benchmark invokes 50 parallel clients, with a keep alive of 1, using a 3-byte payload.)

I am quite surprised at the performance penalty here for full durability.

Aside from the numbers, there are a couple of other interesting things I noticed during the benchmarking (note that the full benchmark suite was run, not just SET):

- In case 2, the dump.rdb file was 40k. While in case 3, 4, and 5, appendonly.aof was 1.9M. If you're going to to use appendonly in production, be sure to consider the additional storage requirements.
- In every case where appendonly was enabled, I received the following error many times as the test was completing (note that this did not occur during the SET operations):

Error writing to client: Broken pipe

To be fair, this is RC, not GA.

* All data easily fit into memory during these tests.
* The FusionIO used was: 160 GB SLC ioDrive
* XFS was the filesystem
* redis version was a git clone at Thu Dec 10 10:41:50 PST 200

Entry posted by Ryan Lowe | 7 comments

Add to: | | | |