Mats Kindahl of Oracle is lead developer of MySQL Fabric
MySQL Fabric is an integrated framework for managing farms of MySQL servers with support for both high-availability and sharding. Its development has been spearheaded by Mats Kindahl, senior principal software developer in MySQL at Oracle.
Mats is leading the MySQL Scaling and High-Availability effort covering the newly released MySQL Fabric and the MySQL Applier for Hadoop. He is also the architect and implementer of several features (mostly replication features), including the row-based replication available in 5.1 and the binary log group commit available in MySQL 5.6. Before starting MySQL he earned a doctoral degree in the area of automated verification of distributed systems and worked with implementation of C and C++ compilers.
He’ll be presenting at next month’s Percona Live MySQL Conference and Expo 2014 in Santa Clara, Calif. and said he was eager to speak with people following his session on the topic — especially those with real-life experience managing large farms. I had a conversation with Mats via Skype recently and here’s what he had to say.
Tom: Hi Mats, thanks for chatting with me today. Your session is titled “Sharding and Scale-out using MySQL Fabric.” What is MySQL Fabric and why did your team decide to create it?
Mats: MySQL Fabric is an extensible framework to manage large farms of MySQL servers. Currently we’ve added support for high-availability and sharding. That’s where we are right now.
MySQL has always had the “15 Minute Rule,” meaning that you should be able to install the software and have it up and running within that time frame. This aspect of ease of use is actually in my opinion part of the success of MySQL. It’s focused on being a flexible and versatile tool.
Given that we have this rule, it was easy to install and easy for people to get up and running on it. But now the trend is to use many servers in clusters — or what we like to call “farms” instead of “clusters” to avoid confusion with products such as MySQL Cluster, which is a different solution.
Using these large farms changed these rules to some extent and it’s not that easy to get up and running anymore if you have a lot of servers. It’s still not a problem with a single server or a few servers, but it becomes complicated once you start to add a large number of servers and try to change configuration and manage it dynamically.
What’s needed here is some kind of general tool for managing big farms of servers. There are several tools available now in various places but each has a very special purpose focusing on a single aspect or a few aspects of managing a big system.
With MySQL Fabric, the focus is on building a framework focused on ease of use for managing and using large farms — treating them as one big distributed database: A federated database, in a manner of speaking.
The immediate goal with MySQL Fabric is to build a framework and implement some of the more critical features people need as extensions. We implemented one extension for handing high availability using the traditional master-slave setup — where you keep a single master and a number of secondaries as slaves. Once the master dies for some reason, the users have historically had to promote one of the slaves to be the new master — with MySQL Fabric, that is automatic.
In addition to failing over the slaves, you had to failover the application or applications too. And this is something that has not been fully automatic. What we have done is to make connectors an integral part of the entire framework, and that way when you are working with a MySQL Fabric farm the application doesn’t have to configure or manage failovers manually. The connectors will automatically realize something is wrong, connect to the MySQL Fabric to get information on what happened, and then preform the failover automatically, updating it’s information about the state of the farm in a way that is almost transparent to the application.
Tom: So then MySQL Fabric is designed to make MySQL much easier to work with – especially for large, data-intense installations.
Mats: Yes. Basically we saw that MySQL was difficult to use in larger setups. And there were no easy to use, flexible tools available that actually could handle what MySQL Fabric does. Most other tools are either special purpose or very limited in being able to manage specific aspects of managing large farms.
There are tools for sharding, there are tools for high availability – there are tools for other things. We want to have something that can actually be used as a framework and can be extended, so that anybody who wants to add special purposes for their setup will be able to add this to the Fabric to manage any specific needs they have.
We couldn’t find anything before like that so we decided to focus on building something that could.
Tom: Is MySQL Fabric free?
Mats: It is available to the community now. Exactly like MySQL Server, MySQL Fabric is open source. It is licensed under version 2 of the GNU General Public License (the “GPL”). And you can currently download it from dev.mysql.com where you’ll also find documentation.
Tom: When was MySQL Fabric’s first release?
Mats: We first released it publically last autumn at MySQL Connect and released another version in December just before Christmas with some new additions and some new features.
Tom: How many users would you say there are now?
Mats: As far as we know, no one have deployed it but we know that people are experimenting with it. We haven’t released it as a product yet — it’s in a pre-GA state. I’d call it a beta right now and we are working on the public bug reports
Tom: Your session at Percona Live next month is going to focus on MySQL Fabric – what can attendees expect and what will they walk away with afterward?
Mats: I’ll basically be talking about the architecture of Fabric, and that of course is centered around how you normally build a big system. You’ll also see how to use MySQL Fabric as a tool to setup a new farm to use for both high availability and for sharding.
Tom: Who should be taking a closer look now at MySQL Fabric? And who perhaps can wait a bit more?
Mats: We have focused on getting a system that can manage very large farms – so we are focusing on trying to eliminate so-called middleman approaches using proxies to increase performance. You probably won’t need anything like this if you have a simple setup with one server, or static site with one server and a few other servers that replicate for backups or reporting. But once you start adding servers or move around the servers more dynamically, then this is a tool that will be very useful. It is also valuable for users who have a small set of servers that they want to add high-availability to, since it simplifies administration.
Tom: What is the learning curve?
Mats: The focus is to be easy to use. People should be able to use it even for small setups. But it is architected in such a way that you should be able to use it even for very large setups, too.
Tom: Aside from your session, what are you interested in attending at next month’s conference?
Mats: I’m quite interested in anything related to high-availability and scalability — especially real-life experience from people managing large farms. There is a lot of value in seeing and understanding what kind of problems people run into in practice. Because it’s never what you see in a theoretical presentation on high-availability — how you manage it. It’s always the details that make a difference.
Two sessions in particular: One is Chip’s session (Chip Turner of Facebook) on asynchronous MySQL, which looks very interesting. And also, given the new improvements of replication in MySQL 5.6, Yoshinori’s (Yoshinori Matsunobu) session on Facebook’s usage of 5.6 is probably going to be very interesting.
There are also several sessions on sharding. Tumblr has one on their use of Jetpants that also looks interesting. I’ve attended that one before but it’s always interesting to talk with them and ask questions.
Additionally I saw that Justin (Swanhart) has a presentation on using Shard-Query, which looks very interesting.
This conference always provides a very diverse audience of users and developers. You always see something new or learn of something that’s coming down the road. And you are also able to talk with people about their real experiences — both good and bad –with using MySQL. So it’s useful for me as a developer to learn about the good parts, because it’s encouraging, but also the bad parts, so we can fix them.
Tom: Mats, thank you very much for speaking with me today. I look forward to meeting you next month in Santa Clara!
Mats: Thank you Tom! It was very nice speaking to you and I look forward to meeting you, and of course all the MySQL developers and users, at the conference next month.
* * *
The Percona Live MySQL Conference and Expo 2014 runs April 1-4 in Santa Clara, Calif. Use the code “SeeMeSpeak” on the registration page and save 10 percent.
UPDATE: April 7, 2014
Here’s a short interview I had with Mats following his session at Percona Live last week:
