One of the tasks within Percona Remote DBA is to ensure we have reliable backups with minimal impact. To accomplish this, one of the tools our team uses is called mydumper. We use mydumper for logical backups because of several nice features. Some of them are:
- multithreaded, producing very fast backups compared to mysqldump
- almost no locking, if not using non innodb tables
- built-in compression
- separate files for each table, making it easy to restore single tables or schema, and the possibility to hardlink files reducing the space needed to keep history of backups. Also this feature give you the possibility to restore with more than one thread.
The mydumper project was started at the beginning of 2009 by Domas Mituzas, and a few months ago we started collaborating and adding new features to improve performance and usability.
And now we are glad to announce our second release: mydumper 0.6.1. You can download the source code here. It is highly recommended to upgrade it if you are using 0.6.0 as we fixed and improved the new less-locking feature.
New features in the mydumper 0.6 series:
- Consistent backups with less locking
This new feature consists of locking all non-innodb tables with the dumping threads at the beginning so in this way we can unlock the flush tables with read lock earlier and no need to wait until all non-innodb tables were dumped. You can take advantage of this feature when you have large archive or myisam tables. - File size chunks
Now you can split tables dump into different files with fixed size. This is usefull to reduce storage capacity needed to keep history backups by using hardlinks. Think on big “log” tables or tables where old data didnt change, now you will be able to hardlink back those chunks. - Metadata Locking
Added new option –use-savepoints to reduce metadata locking issues while backup is running.
Fixed Bugs in the mydumper 0.6 series:
mydumper 0.6.0
- #1250269 ensure statement is not bigger than statement_size
- #827328 myloader to set UNIQUE_CHECKS = 0 when importing
- #993714 Reducing the time spent with a global read lock
- #1250271 make it more obvious when mydumper is not successful
- #1250274 table doesnt exist should not be an error
- #987344 Primary key name not quoted in showed_nulls test
- #1075611 error when restoring tables with dashes in name
- #1124106 Mydumper/myloader does not care for 0-s in AUTO_INCREMENT fields
- #1125997 Timestamp data not portable between servers on differnt timezones
mydumper 0.6.1
- #1273441 less-locking breaks consistent snapshot
- #1267501 mydumper erroneously always attempts a dummy read
- #1272310 main_connection keep an useless transaction opened and permit infinite metadata table lock
- #1269376 mydumper 0.6.0 fails to compile “cast from pointer to integer of different size”
- #1265155 create_main_connection use detected_server before setting it
- #1267483 Build with MariaDB 10.x
- #1272443 The dumping threads will hold metadata locks progressively while are dumping data.
Note: #1267501 fix is important for any galera cluster deployment, because of this bug #1265656 that was fixed and released in Percona XtraDB Cluster 5.6.15-25.3.
Great job!
Unfortunately seems still broken install for CentOS 6.x builds up to 6.5 https://bugs.launchpad.net/mydumper/+bug/1157113 🙁
A very anticipated release. It’s great to see that the mydumper project gets renewed
On Ubuntu Server 12.04 LTS, you need to get the dependencies listed here before MyDumper would build successfully without any errors: Fast parallel MySQL backups and imports using MyDumper.
The patch is no longer needed with 0.6.1, so ignore that part.
George this release is focused mainly in new features, we are working on make it easier to compile on latest versions.
Thanks Max very much appreciate the effort as it’s good to see mydumper development continuing as it’s definitely a useful tool !
Also any chance of getting back the mydumper.org domain, seems to missing in action so I uploaded the old docs for reference at http://centminmod.com/mydumper.html right now.
Looking forward to being able to use mydumper on CentOS 6.x again with MySQL 5.5/5.6, MariaDB 5.5/10 and Percona 5.5/5.6 🙂
Does it work on Windows? If so, what caveats exist?
Rick, I’ve never tried, cygwin could have all necessary packages but not sure
XtraBackup is not maintained on Windows, for some reason. And you mention multiple threads, which gets tricky in Windows.
Any chance off getting deb/rpm packages for this?
Nils, not at this time, but we have plans for it in the near future
Max, how near is the near future for .deb ? 🙂
Thanks
Kubilay
Is there a reason that –less-locking isn’t a default setting? Makes me curious about downsides, if any. Thx!
this is very good, one thing i want to ask,is this backup result can be use for creating slave database?
Sure, you can get the binlog position from the metadata file
Do we still get performance benefit using mydumper over mysqlpump? Did anyone checked?