August 23, 2014

Announcement of Percona XtraDB Cluster (alpha release)

I am happy to announce the availability of alpha release of our new product Percona XtraDB Cluster.

Percona XtraDB Cluster is High Availability and Scalability solution for MySQL Users and based on Percona Server 5.5.17

Percona XtraDB Cluster provides:

  • Synchronous replication. Transaction either commited on all nodes or none.
  • Multi-master replication. You can write to any node.
  • Parallel applying events on slave. Real “parallel replication”.
  • Automatic node provisioning.
  • Data consistency. No more unsyncronised slaves.

Percona XtraDB Cluster is fully compatible with MySQL or Percona Server in the following meaning:

  • Data compatibility. Percona XtraDB Cluster works with databases created in MySQL / Percona Server
  • Application compatibility. There is no or minimal application changes required to start work with Percona XtraDB Cluster

Percona XtraDB Cluster is based on Percona Server with XtraDB and includes Write Set REPlication patches. It uses the Galera library, version 2.x, a generic Synchronous Multi-Master replication plugin for transactional applications.

The Galera library is developed by Codership Oy company and
Galera 2.x supports such new features as:

  • Incremental State Transfer, especially useful for WAN deployments
  • RSU, Rolling Schema Update. Schema change does not block operations against table

Please note, this is an alpha release, not suitable for production yet. Bugs and some rough edges are expected. We are encouraging you to try and play with it in testing environment and report us your feedback and experience.

Links:

Previous posts on this topic:

About Vadim Tkachenko

Vadim leads Percona's development group, which produces Percona Clould Tools, the Percona Server, Percona XraDB Cluster and Percona XtraBackup. He is an expert in solid-state storage, and has helped many hardware and software providers succeed in the MySQL market.

Comments

  1. Hi Vadim and team

    Congratulations to everyone for launching this! This will be a game changer in the MySQL world.

  2. Steve Jackson says:

    Now this rox. Certainly gonna have a play…

  3. marrtins says:

    Wow, cool! Will try this some day.

  4. Andy says:

    This is huge. Congrats!

    Will this work with handlersocket? I’m using Percona Server 5.5 with handlersocket.

  5. pcrews says:

    I’ll be writing more about the testing we’ve been doing soon, but it’s looking quite solid so far!

  6. Louis says:

    Sounds terrific. Can’t wait to try that. Thanks!

  7. nowgoo says:

    Great work, Congrats! I’ll test it soon.

  8. jackbillow says:

    good job,congratulations. it become more and more for mysql with ha, replication issue and many node write nodes.

  9. I agree with Henrik. This is a game changer. To have a MySQL server with synchronous replication built in resolves (most of) the need for an external HA solution, of slave promotion issues, slave lag issues etc. Am anxious to see how this goes.

  10. vineet says:

    Great News, Looking forward for stable release soon.
    Especially i am looking for Galera with XtraDB storage engine as Galera with InnoDB have problems like increase in deadlock.

  11. Fraj says:

    Good News, i will try it

  12. Vineet: XtraDB will behave exactly the same wrt deadlocks. You should write your code so that deadlocked transactions are handled correctly (usually just re-start them). If you can’t do that, you can choose to write to a single master to get same behavior as with a standalone innodb. (Using other nodes for redundancy or as read-only nodes.)

  13. vineet says:

    Thanks Henrik for your input.

    I ran sysbench against Galera nodes and got huge no. of deadlocks. Figures are below (its a very small infrastructure based test)

    Galera with Single Node
    transactions: 133423 (148.13 per sec.)
    deadlocks: 0 (0.00 per sec.)
    total time: 900.7191s

    Galera with 2 Node
    transactions: 271778 (301.85 per sec.)
    deadlocks: 8544 (9.49 per sec.)
    total time: 900.3745s

    Galera with 3 Node
    transactions: 303919 (337.55 per sec.)
    deadlocks: 7084 (7.87 per sec.)
    total time: 900.3613s

    As Percona guys done lots of changes at storage engine level for performance so i was thinking of improvement in deadlock cases.

  14. Hi Vineet: interesting numbers. Maybe you had much smaller table than I used in my own sysbench tests (thus causing more collisions). See http://openlife.cc/blogs/2011/august/more-galera-lessons-parallel-slave-out-order-commits-and-deadlocks for same benchmark I did. Even in the worst case I have less than 0.25% of deadlocks and not at all under light load.

    It’s not the amount of deadlocks that worries me, but that they might happen at all. I’m not sure all code out there handles deadlocks properly, so unless people review their code they’ll start seeing spurious errors. But otoh the frequency of deadlocks is so small maybe they won’t notice that they have a problem. (Codership guys say that nobody has complained about this so far…)

  15. vineet says:

    Henrik, I already follow your blogs :) .

    But i can’t compare my results with yours as i do tests on very small scale like 4~8 GB RAM.

    Might be right time for me to revisit my tests and update Codership guys for the same.

  16. Vineet: Actually, it is interesting information to see the amount of deadlocks increase when you have a smaller table. You said you have 4-8GB of RAM, but how many rows in your sysbench table, and how many tables?

  17. vineet,

    What parameters did you use for Run ?

    Sysbench by default creates a pretty serious hotspot. Also note the cross node deadlock detection in Percona XtraDB Cluster is different from the one for single note – it is essentially optimistic locking as such no wait happens in case 2 transactions update the same row but on commit one of transactions will fail.

    In sysbench this can be easily exposed yet it is often less of the problem in the real world unless you’re having some very hot rows, such as counters etc.

  18. vineet says:

    @Henrik & Peter

    I used below parameters for my test.

    sysbench –num-threads=99 –max-time=900 –max-requests=1000000 –test=oltp –mysql-host=10.83.14.40,10.83.14.94,10.83.14.90 –mysql-user=root –mysql-password=rootpass –mysql-table-engine=InnoDB –mysql-engine-trx=yes –max-requests=0 –oltp-table-size=1000000 –oltp-auto-inc=off run

    And i have started my test with Single table (Which is default in SysBench).

    That will be other scenario for me to test Galera with different no. of tables.

  19. Ok, so you have 20x less rows than I had, and get roughly 60x more deadlocks. Interesting comparison :-)

  20. vineet,

    One thing you might want to try to change in addition to table size is –oltp-dist-type=uniform this will use uniform distribution touching all rows with same frequency.

  21. Alex says:

    Vineet,

    Looking at how the deadlock rate actually _decreases_ with the number of nodes added and looking at the number of concurrent connections, I’d suggest that you have oversaturated benchmark. Transactions simply sit there too long waiting for CPU – hence have a higher chance of a conflict. I suspect that if you decrease -num-threads parameter, you’ll get better scores and less deadlocks.

  22. vineet says:

    @Henrik: Thanks for your input.

    @Peter: Thanks for your suggestion, I’ll try this out.

    @Alex: My testing is in initial phase, I have many scenarios to test. Will try your suggestion and update you guys accordingly.

  23. So Flashcache is enabled. Cool.
    How can I turn this off. I’m getting:

    120111 14:10:39 [Note] Flashcache bypass: disabled
    120111 14:10:39 [Note] Flashcache setup error is : ioctl failed
    Thanks

  24. Vadim says:

    Mark,

    It says it is disabled. You can just ignore these [Note]

  25. Developer says:

    What is the difference (in features, limitations, etc…) between Xtradb cluster compared to MySQL cluster?

    And, if one is already deployed on MySQL cluster, does switching to Xtradb cluster at this point make sense or is it too immature?

  26. Developer

    MySQL Cluster is very different, so it is hard to make correct comparison.

    There is my opinion, which may be not full

    Percona XtraDB Cluster
    ● It is based on widely used XtraDB/InnoDB engine.
    ● It is almost drop-in replacement for existing installation of MySQL replication
    ● Minimal or NO application changes
    ● Less additional knowledge for DBA is required to run XtraDB Cluster than NDB Cluster.

    MySQL NDB Cluster is totally different engine than MySQL Replication
    ○ Requires changes in hardware
    ○ Requires application changes
    ○ Requires very knowledgeable DBA
    ○ Very hard for diagnostic and troubleshooting
    ○ expensive to deploy and maintain

    Percona XtraDB Cluster has:
    ○ Minimal entry barrier for one who switches from MySQL Replication or for new deployments.

    Also please see
    http://www.mysqlperformanceblog.com/2012/01/11/making-the-impossible-3-nodes-intercontinental-replication/

    However I advise to make changes only if you need it.
    If you currently run MySQL Cluster and you happy with it, you should stay with it.

  27. Developer says:

    Vadim, thanks for the info. To follow up, are there are features provided by mysql cluster that is NOT available in xtradb cluster?

    EG: MySQL cluster supports online DDL, whereas it seems that online DDL is not supported by xtradb cluster?

    http://www.percona.com/doc/percona-xtradb-cluster/limitation.html
    DDL statements are problematic and may stall cluster. Later, the support of DDL will be improved, but will always require special treatment.

    EG: MySQL cluster 7.2 supports join-push down with adaptive query localization. Is that supported by xtradb cluster?

    Also, what is the maximum # of data nodes and sql nodes supported by xtradb cluster? The documentation does not mention an upper limit on the # of nodes, whereas ndb is limited to 48 data nodes and 256 total nodes.

  28. Developer,

    Yes, there are features in MySQL Cluster that is not available in XtraDB Cluster. Again, it is totally different products.

    MySQL Cluster is designed for scale out, XtraDB Cluster is not.

    Online schema change is work in progress in XtraDB Cluster and some solution will be available later.

    join-push is not applicable to XtraDB Cluster, as all queries in XtraDB Cluster are executed locally on node where query was initialized.

    There is no software limitation for amount of nodes, but there will be logical limitation when sending transaction to all N nodes will be to expensive and will kill your performance. The number N is individual for each setup and depends on hardware, network connections, workload.

  29. Pretty good discussion here on NDB vs XtraDB clusters.

    One point that didn’t come up is that NDB is a real-time database. You can be guaranteed that for some threshold of N, 99.9…% of your queries will have a response time less than N. For simple key value operations N is 1 or 2 ms. For InnoDB (ie XtraDB Cluster) you will not see as stable performance as this, although if your data fits in memory it will be pretty good in practice.

    Also, it is said that NDB is totally different and therefore very difficult to understand, and you will need application changes if migrating from a standard MySQL server to an NDB Cluster. This is all true. Otoh in your case if you are already running on MySQL Cluster, the same might be true the other way: you might be better of just staying where you are.

  30. Max says:

    What Vadim and Henrik are saying is very true, MySQL Cluster is a completely different animal. I guess the two main differences come from MySQL Cluster doing two things:

    1) Separation of MySQL Server and Storage Engine; meaning that the Storage Engine is run as a separate process which can then be on a different machine. This adds great flexibility as you can add MySQL Servers and data nodes independently but adds complexity

    2) Data partitioning; MySQL Cluster partitions the data automatically between the data nodes which means that it’s great for scale-out (as Vadim mentioned) as write queries scale as well (with more than 2 data nodes). This is a great benefit compared to most other solutions. The downside with this is again complexity, both for the administrators and internal complexity. The commit protocol for example has so many messages that cluster is pretty much useless on a geographical scale. What you can do of course is combine MySQL Cluster with replication, but then you’re back to asynch.

    So all in all, either you design your application for MySQL Cluster or you don’t. It is never (ok, hardly ever) a drop in replacement.

Speak Your Mind

*