Comments on: Using netcat to copy MySQL Database http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/ Percona's MySQL & InnoDB performance and scalability blog Sat, 11 Feb 2012 16:45:54 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: eaglesflyhigh http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-698352 eaglesflyhigh Fri, 18 Dec 2009 23:06:27 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-698352 Brains and Genius,respect to you all....i read almost everything on this site but i still need someone to put me through... im of good intention to life everything i learn, im a fast learner and i'd like to meet someone who 'd be so kind and humble to please be my teacher/master.. to teach me some few thing which i could develop later on my own we could use yahoo or msn messenger for the training session…..deepspeed_772000@yahoo.com looking forward to meet you. thanks simplee me Brains and Genius,respect to you all….i read almost everything on this site but i still need someone to put me through… im of good intention to life everything i learn, im a fast learner and i’d like to meet someone who ‘d be so kind and humble to please be my teacher/master.. to teach me some few thing which i could develop later on my own we could use yahoo or msn messenger for the training session…..deepspeed_772000@yahoo.com
looking forward to meet you. thanks
simplee me

]]> By: eaglesflyhigh http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-698346 eaglesflyhigh Fri, 18 Dec 2009 23:00:44 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-698346 Good day, i need a good ip changer and how do i get more bandwidth from my server .....deepspeed@yahoo.com Good day, i need a good ip changer and how do i get more bandwidth from my server …..deepspeed@yahoo.com

]]> By: James Byers http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-660021 James Byers Wed, 30 Sep 2009 05:56:28 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-660021 If using Ken's rsync example and you're root on both sides, you'll likely need: uid = root gid = root above [db] in the rsync config file to avoid permission problems. If using Ken’s rsync example and you’re root on both sides, you’ll likely need:

uid = root
gid = root

above

[db]

in the rsync config file to avoid permission problems.

]]> By: Morgan Christiansson http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-586308 Morgan Christiansson Mon, 15 Jun 2009 20:17:40 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-586308 It is possible to use a less expensive cipher with ssh which will speed things up significantly. For example: rsync -e 'ssh -c blowfish' ... scp -c blowfish ... It is possible to use a less expensive cipher with ssh which will speed things up significantly.

For example:
rsync -e ‘ssh -c blowfish’ …
scp -c blowfish …

]]> By: Kevin Burton http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-578387 Kevin Burton Mon, 08 Jun 2009 04:38:01 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-578387 I agree with #1 .... do NOT do this on production data without checksums. You WILL get data corruption. I agree with #1 …. do NOT do this on production data without checksums.

You WILL get data corruption.

]]> By: Ken http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-575777 Ken Fri, 05 Jun 2009 19:58:39 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-575777 RT: agreed. Przemek: 1. You'll find more and more that these days people using MySQL run HA systems (as HA as MySQL gets anyway) for international users and systems, not just a little PHP site in Dubuque. The "night-time trough" is no longer a luxury for a lot of the client base of MySQL. 2. LVM snapshot overhead alone can be 20-50%; startups with limited capex don't want to pay 20-50% more for their storage hardware to support a method of database backup used solely because a working solution isn't available from the DB or engine vendor (explain /that/ to a VC). Established companies don't want to budget in 20-50% more for their expensive SAN setups. We'd all love a few thousand spare TPS or a few 100MB/s of spare throughput, but that money should be going to what makes money, not what loses money. This also doesn't cover the overhead of saturating the disk subsystem with the *actual copy* with tar/nc or rsync. Been there. Done both. Bought the T-shirt. 3. Absolutely agree; the many other benefits of a slave makes the cost worthwhile. LVM is a very useful tool, but the fact that it is often represented as a panacea by the MySQL community (not singling you out) does a disservice to those relying on that information (who then have it blow up in their face). IMHO; apologies for the rant. -- Ken. RT: agreed.

Przemek:

1. You’ll find more and more that these days people using MySQL run HA systems (as HA as MySQL gets anyway) for international users and systems, not just a little PHP site in Dubuque. The “night-time trough” is no longer a luxury for a lot of the client base of MySQL.

2. LVM snapshot overhead alone can be 20-50%; startups with limited capex don’t want to pay 20-50% more for their storage hardware to support a method of database backup used solely because a working solution isn’t available from the DB or engine vendor (explain /that/ to a VC). Established companies don’t want to budget in 20-50% more for their expensive SAN setups. We’d all love a few thousand spare TPS or a few 100MB/s of spare throughput, but that money should be going to what makes money, not what loses money. This also doesn’t cover the overhead of saturating the disk subsystem with the *actual copy* with tar/nc or rsync. Been there. Done both. Bought the T-shirt.

3. Absolutely agree; the many other benefits of a slave makes the cost worthwhile.

LVM is a very useful tool, but the fact that it is often represented as a panacea by the MySQL community (not singling you out) does a disservice to those relying on that information (who then have it blow up in their face). IMHO; apologies for the rant.
–
Ken.

]]> By: Przemek http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-575731 Przemek Fri, 05 Jun 2009 19:01:29 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-575731 1. You do it at off peak time. 2. IMHO if your system dies only because of LVM snapshot overhead - you're far too close to your machine's disk I/O limits. 3. IMO always the best solution is to have a spare slave used only for backups, some periodic heavy queries, etc. 1. You do it at off peak time.
2. IMHO if your system dies only because of LVM snapshot overhead – you’re far too close to your machine’s disk I/O limits.
3. IMO always the best solution is to have a spare slave used only for backups, some periodic heavy queries, etc.

]]> By: Richard Thomas http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-575728 Richard Thomas Fri, 05 Jun 2009 18:49:39 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-575728 As its even been reported on this blog, the LVM snapshot penalty can be extreme, on a overly busy server (and yes I understand thats a problem of its own) starting up an LVM snapshot can cause a cascading failure of the system as it struggles to keep up. As its even been reported on this blog, the LVM snapshot penalty can be extreme, on a overly busy server (and yes I understand thats a problem of its own) starting up an LVM snapshot can cause a cascading failure of the system as it struggles to keep up.

]]> By: Przemek http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-575719 Przemek Fri, 05 Jun 2009 18:42:40 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-575719 Instead of shutdowning master just use LVM snapshot method to create a slave or backup. This way you only need a few secs of read lock. If you don't use LVM partitions I suggest to start using them (optimal separate ones for data and binlogs). Instead of shutdowning master just use LVM snapshot method to create a slave or backup. This way you only need a few secs of read lock.
If you don’t use LVM partitions I suggest to start using them (optimal separate ones for data and binlogs).

]]> By: Richard Thomas http://www.mysqlperformanceblog.com/2009/05/31/using-netcat-to-copy-mysql-database/comment-page-1/#comment-575704 Richard Thomas Fri, 05 Jun 2009 18:18:44 +0000 http://www.mysqlperformanceblog.com/?p=697#comment-575704 Rsync is great for live replication copies also 1. rsync with database running, depending on last rsync can take a while 2. Flush tables/cache, rsync again, won't take nearly as long 3. shutdown master 4. rysnc one final time, on a 10gig db only takes a couple seconds 4. restart master, downtime only a few seconds and you have a fresh copy to use for starting a slave back up Of course you need to delete the proper log files depending on your setup and how you store your log files. Rsync is great for live replication copies also

1. rsync with database running, depending on last rsync can take a while
2. Flush tables/cache, rsync again, won’t take nearly as long
3. shutdown master
4. rysnc one final time, on a 10gig db only takes a couple seconds
4. restart master, downtime only a few seconds and you have a fresh copy to use for starting a slave back up

Of course you need to delete the proper log files depending on your setup and how you store your log files.

]]>