… our Benchmarks Launchpad
The graphical results:
Throughput (more is better)

Threads MariaDB 5.3.4 MySQL 5.5.20 Ratio
1 252 …256 3710 4131 0.8980876

I was surprised to see that MariaDB shows 20-30% worse throughput.
It seems many changes resulted …

… and is a conspicuous omission, so it would probably be helpful for the MariaDB team to publish their benchmarks. Here is one place where some … are hints at extensive benchmarks: http://www.odbms.org/blog/2011/09/mariadb-the-new-mysql-interview-with-michael-monty-widenius/ The obvious question is if…

… would be mainly of academic interest. These results highlight the real challenge MariaDB poses for users right now – do you want improved concurrency MySQL 5.5 provides or do you want Optimizer Changes MariaDB team developed ?

If your workload is simple (sysbench like) MySQL 5.5 …

…XtraDB engine which scales pretty well in both 5.1 and 5.5.

Now about fairness:
MariaDB 5.3 is RC for the next production ready MariaDB‘s flagship … the current pre-production product is 5.6.

So there we compare the future MariaDB with the current MySQL 5.5, and probably more fair would be to …

MariaDB 5.3.4 runs ontop XtraDB 1.0.17-13.0 which in turn is based on InnoDB 1.0.17 from MySQL 5.1. So indeed …5.1 or Percona Server 5.1.59-13.0 would be proper targets for a comparison with MariaDB 5.3.4.

As pointed out by Peter, sysbench runs simple queries only where MariaDB cannot demonstrate …

I agree with comparison of MariaDB 5.3 vs MySQL 5.5. Comparing it to MySQL 5.1 is really useless (at last to me). I hope MariaDB 5.5 get ready before MySQL 5.6.

> [..] MariaDB with the current MySQL 5.5, and probably more fair would be to compare MariaDB 5.3 vs MySQL 5.6.

If your goal was to compare prerelease products, sure, but that wasn’t the goal that you established in the first paragraph — “check how all improvements affect general performance”.

> I was surprised to see that MariaDB shows 20-30% worse throughput.

Might it be that this actually compares the scalability of 5.1 versus 5.5? I think it would be more fair to compare MariaDB 5.3 against MySQL 5.1 (which 5.3 is based from). Perhaps, also use a …

… benchmarks

What about re-running benchmarks with 5.3.4 new optimizers disabled ? Would be interesting to see benchmarks for other ‘common type’ of work loads and more complex query work loads as well.

But I agree with Peter, I hope MariaDB on 5.5. code base comes sooner rather than later!

Actually MariaDB team ran sysbench several times before releasing 5.3.4 and we did not have such a degradation as Vadim had, far from it. Our configuration was different though. Axel will probably write about his results.