XtraBackup: Open Source Alternative for Innodb Hot Backup call for ideas
For long time as main backup solution for MySQL on Linux we have been using LVM snapshots. Performance concerns from performance critical envinronment caused us to do LVM performance research which showed horrible results forcing us to look at more performing alternatives.
Innodb Hot Backup is a good working solution but it is not Open Source and so we can’t ensure it will support all XtraDB features – extra undo slots, rollback segments etc.
After considering all alternatives we decided to develop Percona Xtrabackup tool, which will use same backup approach as InnoDB Hot Backup, that is taking online copy of InnoDB table files and transactional logs. This is going to be fully OpenSource, GPL licensed backup solution for MySQL (MyISAM and InnoDB for now).
We gathered basic ideas here http://www.percona.com/docs/wiki/percona-xtrabackup:start, and for the first stage it will be just full backup of files. For next stages we are looking to implement advanced functionality like incremental backup (copy only changes from last backup),table-level backup (copy only specified tables), streaming and parallel copying. Source code will be posted to Xtrabackup Launchpad project (https://launchpad.net/percona-xtrabackup/)
We call for more ideas you would like to see implemented (please leave them in comments). We got sponsorship for the base implementation work, but we’re still looking for sponsorship for additional features, contact us http://www.percona.com/contacts.html if you are interested. Percona will also significantly invest in this project.
del.icio.us
digg8 Comments
Hi,
Great to hear! Here are a few ideas:
* I was always upset with the fact the restore (ibbackup –apply-log) restores the ibdata and log files *in the same path* as the backup files. This made for harder work when disk space is limited.
The backup stage allows for different paths, which is nice. If you could have this simple feature – it would make a great difference.
* The restore part has the property of not letting you know how well it is progressing. I mean, there’s the unpacking of the ibdata file – great. But then it start “sppliting logs”, over and over and over, each time counting 0..100. You just never know if you should keep waiting for an hour or 5 hours (I got 5 hours a couple of weeks ago, and it was frustrating)
So – a progress indicator would be great!
* Obviously – if you could bundle the schema along with the data, it would be beneficial. It happened to me once, that due to a programmer’s mistake, the schema was changed while (or just before) creating the schema, but after I’ve dumped the schema to the new backup machine. So after hours of restoring from backup – I got errors as the schema was incorrect.
Good luck with development
Comment :: February 25, 2009 @ 12:05 am
Typo (how did I get it so wrong?) above:
“spliting logs” ==> “applying logs”
Comment :: February 25, 2009 @ 12:07 am
Wow, sounds great. Finally some alternative to innodb hotbackup.
Don’t get me wrong, its a great tool but too expensive for my use case and there hasn’t been much development anymore.
Comment :: February 25, 2009 @ 5:01 pm
Yeah, sounds great!
few ideas:
- mysql/innodb is able to use raw partitions. as there are no .ibd files in this case, it would be nice to backup these as well. for those who made the decision to enable a raw partition, this could be a way to restore the backup using “normal” ibd files instead.
- having a small file indicating the binlog position along with the backup files could help enabling replication. (another way would be adding a “flush tables”, but if one has to use a hot backup tool, its unlikely that one wants to flush tables
Comment :: February 25, 2009 @ 6:18 pm
Shlomi, Bernd
Great, thank you for ideas, we added them to the list!
Comment :: February 25, 2009 @ 11:21 pm
Thanks. I’m going to try your software on my DB server
Comment :: March 11, 2009 @ 5:24 am
I am using Zmanda, and what I like is easy configuration and scheduling of backups, and reports.
It gives you a good visibility on what is going on with your backups.
Zmanda is not a backup tool but more precisely a backup managing tool,
it uses all known backup solutions already existing.
I dont know if it is good to have this features in Xtrabackup or just think if it would be possible to integrate Xtrabackup into Zmanda like innodb hot backup is. Just some ideas.
Cheers
Claudio
Comment :: May 18, 2009 @ 7:22 am
Claudio,
We are open to integrate it, but we can’t force Zmanda to add support. If you are user of Zmanda you probably can send feature request to them to add support of xtrabackup.
Comment :: May 18, 2009 @ 9:19 am