Comments on: Compression for InnoDB backup

By: Snarky

Snarky — Tue, 23 Jun 2009 19:38:46 +0000

The decompression speed computation is deceiving. You should divide by the size of the uncompressed data not the compressed one because the better the compression the worse the decompression speed will look when it is not the case.

By: slavik

slavik — Sat, 21 Mar 2009 07:27:07 +0000

I found that LZMA can’t scale, with -1 it can use only 1 thread, with -5 (or bigger) only 2 worker threads.
“Sets multithread mode. If you have a multiprocessor or multicore system, you can get a increase with this switch. 7-Zip supports multithread mode only for LZMA compression and BZip2 compression / decompression. If you specify {N}, for example mt=4, 7-Zip tries to use 4 threads. LZMA compression uses only 2 threads.” http://www.bugaco.com/7zip/MANUAL/switches/method.htm
In my tests on amd 9950 with 2gig of ram: 4 mb/s compression, and about 8 mb/s decompression.
I think it’s results of terrible optimization of unix port

By: Vadim

Vadim — Thu, 19 Mar 2009 16:33:01 +0000

slavik,

it is Dell PowerEdge R900, 4x quadcores

vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Xeon(R) CPU E7320 @ 2.13GHz
stepping : 11
cpu MHz : 2127.881
cache size : 2048 KB

with 32GB of RAM

By: slavik

slavik — Thu, 19 Mar 2009 05:24:34 +0000

Vadim,
too ugly results for lzma, can u post system spec?
I test compress/decompress on windows 7zip, quad core amd 9500 with 8gb ram, so I have decompress speed 20 mb/s (too close for regular old 160gb PATA drive speed) and compression (fast mode) speed 8mb/s.
I will try later on a similar system under Linux, and post results.

By: Steven Roussey

Steven Roussey — Thu, 19 Mar 2009 04:43:50 +0000

lzjb

Reminds me of discussions in this post:

ZFS & MySQL/InnoDB Compression Update
http://blogs.smugmug.com/don/2008/10/13/zfs-mysqlinnodb-compression-update/

By: Vadim

Vadim — Wed, 18 Mar 2009 05:31:22 +0000

Mark,

for me on Ubuntu 8.10 where I did test – lzop comes linked with LZO v2 libraries.

So even your distributive has lzop with LZO v1 you probably can compile it linked to v2.

By: Mark

Mark — Tue, 17 Mar 2009 23:46:09 +0000

That will teach me to post at 1 in the morning!

Yes, I meant that 7zip does take more resources than gzip, but adding encryption doesn’t add any *more*. As for the performance hit, the machines I use 7zip on have distinct busy and non-busy times, so I can schedule a 25-minute, 7zipped backup during a non-busy time fairly easily. I realize this isn’t necessarily normal for most servers though, for the rest of our off-line backups we use gzip. After looking at these results I’m thinking about lzo instead, especially since it’s less of a hit on a busy machine. Unfortunately, I see that version 2 doesn’t have a nice, gzip-like executable. Since we use 64-bit machines almost exclusively, and v2 promises better performance on 64-bit, does anyone have a link to a command-line archiver that can use lzo v2?

By: Vadim

Vadim — Tue, 17 Mar 2009 18:34:06 +0000

slavik,

I added results for LZMA. with compression level 5 it was not able to finish in 2h, so I stopped that.

By: slavik

slavik — Tue, 17 Mar 2009 16:16:34 +0000

Baron,
Mark meant that using aes encryption does not affect time spent to compression.
This is typical for modern processors, usually because sheduler can’t always use all cores power (cache miss, io bottlenecks, kernel tasks and so on) so small number of cpu resources (but enough for encryption) is always avaiable.

By: Dennis Birkholz

Dennis Birkholz — Tue, 17 Mar 2009 15:25:35 +0000

I think Mark meant that 7z slows the machine so extremly “really” down that additional encryption does not hit the performance any more

I would like to see a lzma benchmark, to but i am not sure if it supports multicore procession.