Comments on: How to calculate a good InnoDB log file size http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/ Everything about MySQL Performance Sat, 21 Nov 2009 05:23:57 -0800 http://wordpress.org/?v=2.8.4 hourly 1 By: Baron Schwartz http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-544846 Baron Schwartz Mon, 20 Apr 2009 22:28:05 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-544846 Yes, I mean innodb_log_file_size. About the slides -- actually that advice was from Peter Zaitsev. He's not giving you all the details you need to understand what he was talking about. You would have had to be in that talk to understand it. Yes, I mean innodb_log_file_size. About the slides — actually that advice was from Peter Zaitsev. He’s not giving you all the details you need to understand what he was talking about. You would have had to be in that talk to understand it.

]]> By: Andy Geers http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-544505 Andy Geers Mon, 20 Apr 2009 11:38:28 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-544505 I'm slightly confused by this article - it seems to match very closely the advice given by the InnoDB developers for the innodb_log_buffer_size setting here: http://www.innodb.com/wp/wp-content/uploads/2007/05/uc2007-innodb-performance-optimization.pdf ... rather than the innodb_log_*file*_size setting that you are talking about. Can you confirm that you definitely mean the setting you're referring to? I’m slightly confused by this article – it seems to match very closely the advice given by the InnoDB developers for the innodb_log_buffer_size setting here: http://www.innodb.com/wp/wp-content/uploads/2007/05/uc2007-innodb-performance-optimization.pdf

… rather than the innodb_log_*file*_size setting that you are talking about. Can you confirm that you definitely mean the setting you’re referring to?

]]> By: John Marc http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-522398 John Marc Sat, 28 Mar 2009 20:55:11 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-522398 Yow! Guess I need to up this! -rw-rw---- 1 mysql mysql 100M Mar 28 16:49 ib_logfile0 -rw-rw---- 1 mysql mysql 100M Mar 28 16:50 ib_logfile1 Using Barons calcs I am looking at 200MB, and Sheens method says 600MB? Yow! Guess I need to up this!

-rw-rw—- 1 mysql mysql 100M Mar 28 16:49 ib_logfile0
-rw-rw—- 1 mysql mysql 100M Mar 28 16:50 ib_logfile1

Using Barons calcs I am looking at 200MB, and Sheens method says 600MB?

]]> By: Vladimir Rusinov http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-461415 Vladimir Rusinov Sun, 01 Feb 2009 23:31:43 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-461415 Hi. Here is translation to Russian (if someone interested): http://greenmice.info/ru/node/92 Hi. Here is translation to Russian (if someone interested): http://greenmice.info/ru/node/92

]]> By: Baron Schwartz http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-434343 Baron Schwartz Sun, 04 Jan 2009 21:37:00 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-434343 That is exactly the point of writing this post. Those guidelines are wrong. Both of them. That is exactly the point of writing this post. Those guidelines are wrong. Both of them.

]]> By: Adam http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-434268 Adam Sun, 04 Jan 2009 20:21:55 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-434268 Heh, I am messing with this now for my research and I'm quite certain it says right in the readme that it should be 25% of your buffer pool size, which should be 50-80% of available memory. To me it seems like 64M is indeed way too low. Heh, I am messing with this now for my research and I’m quite certain it says right in the readme that it should be 25% of your buffer pool size, which should be 50-80% of available memory. To me it seems like 64M is indeed way too low.

]]> By: Jesper Wisborg Krogh http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-421308 Jesper Wisborg Krogh Tue, 23 Dec 2008 04:25:26 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-421308 Thanks a lot - I had considered that as an option as well as well as keeping track of which rows actually need to be changed. It will of course require a bit more work and bookkeeping, but it should be doable and I believe in this case it is the best long term solution. Merry Christmas Thanks a lot – I had considered that as an option as well as well as keeping track of which rows actually need to be changed. It will of course require a bit more work and bookkeeping, but it should be doable and I believe in this case it is the best long term solution.

Merry Christmas

]]> By: Baron Schwartz http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-421302 Baron Schwartz Tue, 23 Dec 2008 04:19:11 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-421302 I think you need to be nicer to your database. Try building the new table in table_new, so you can commit the transaction after every command, and then swapping the table_current and table_new with RENAME TABLE. And now your free advice tokens for the day are all spent :) I think you need to be nicer to your database. Try building the new table in table_new, so you can commit the transaction after every command, and then swapping the table_current and table_new with RENAME TABLE. And now your free advice tokens for the day are all spent :)

]]> By: Jesper Wisborg Krogh http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-421287 Jesper Wisborg Krogh Tue, 23 Dec 2008 04:05:37 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-421287 mysql> SHOW GLOBAL VARIABLES LIKE 'innodb_log_file_size'; +----------------------+-----------+ | Variable_name | Value | +----------------------+-----------+ | innodb_log_file_size | 536870912 | +----------------------+-----------+ 1 row in set (0.00 sec) The error message in the log states that the age of the last checkpoint is 966364465 and the capacity is 966363956. I'm not sure why there capacity is a bit smaller than two times the innodb_log_file_size, but I assume there are some space being used for bookkeeping. It is one particular transaction that seems to trigger the error and that has a total of nine columns: 3 unsigned ints, 4 ints, 1 unsigned tinyint, and one datetime. It is a cache table so the transaction first deletes all rows and then repopulates it following by a couple of table scans to manipulate the data in an ordered update. The total number of rows is 1286113 (determined with SELECT COUNT(*)). The file containing the data for the table is 772 MB. According to SHOW TABLE STATUS the stats is: Data_length: 165838848 Index_length: 219807744 Data_free: 0 Comment: InnoDB free: 399360 kB Other things are going on in the server at the same time. I tried to do a test on our development system and it will run on an idle system with innodb_log_file_size set to 512 MB, so it might also be that we can get around it by trying to reschedule the tasks, however I suspect even so we would be running close to the limit. Which brings me to: is there anyway to monitor whether you are getting close to using all the available space in the innodb log files? mysql> SHOW GLOBAL VARIABLES LIKE ‘innodb_log_file_size’;
+———————-+———–+
| Variable_name | Value |
+———————-+———–+
| innodb_log_file_size | 536870912 |
+———————-+———–+
1 row in set (0.00 sec)

The error message in the log states that the age of the last checkpoint is 966364465 and the capacity is 966363956. I’m not sure why there capacity is a bit smaller than two times the innodb_log_file_size, but I assume there are some space being used for bookkeeping.

It is one particular transaction that seems to trigger the error and that has a total of nine columns: 3 unsigned ints, 4 ints, 1 unsigned tinyint, and one datetime. It is a cache table so the transaction first deletes all rows and then repopulates it following by a couple of table scans to manipulate the data in an ordered update. The total number of rows is 1286113 (determined with SELECT COUNT(*)). The file containing the data for the table is 772 MB. According to SHOW TABLE STATUS the stats is:

Data_length: 165838848
Index_length: 219807744
Data_free: 0
Comment: InnoDB free: 399360 kB

Other things are going on in the server at the same time. I tried to do a test on our development system and it will run on an idle system with innodb_log_file_size set to 512 MB, so it might also be that we can get around it by trying to reschedule the tasks, however I suspect even so we would be running close to the limit. Which brings me to: is there anyway to monitor whether you are getting close to using all the available space in the innodb log files?

]]> By: Baron Schwartz http://www.mysqlperformanceblog.com/2008/11/21/how-to-calculate-a-good-innodb-log-file-size/comment-page-1/#comment-421264 Baron Schwartz Tue, 23 Dec 2008 03:40:25 +0000 http://www.mysqlperformanceblog.com/?p=541#comment-421264 Jesper, I think something else is wrong. Have you *confirmed* that the log files are 512MB? What do you get from this? SHOW GLOBAL VARIABLES LIKE 'innodb_log_file_size'; I think you have a typo in your my.cnf, or you're putting the variables in the wrong my.cnf file, or something like that. Either that, or you're working with blobs that are hundreds of MB or some other extremely unusual workload. Or an InnoDB bug (very unlikely). Either way, if you can't figure out what's wrong after checking these things, I think you probably need expert help :-| Jesper, I think something else is wrong. Have you *confirmed* that the log files are 512MB? What do you get from this?

SHOW GLOBAL VARIABLES LIKE ‘innodb_log_file_size’;

I think you have a typo in your my.cnf, or you’re putting the variables in the wrong my.cnf file, or something like that. Either that, or you’re working with blobs that are hundreds of MB or some other extremely unusual workload. Or an InnoDB bug (very unlikely). Either way, if you can’t figure out what’s wrong after checking these things, I think you probably need expert help :-|

]]>