Comments on: How much memory can MySQL use in the worst case?

By: yingkuan

yingkuan — Mon, 23 Feb 2009 00:38:31 +0000

Erm, correction, I meant to say ‘you should NOT have any innodb related mem usage’

By: yingkuan

yingkuan — Mon, 23 Feb 2009 00:37:15 +0000

Heiðar,

If you are using MYISAM then you should have any Innodb related memory usage like
buffer pool, redo buffer, additional mem pool etc.
you might want to first look into that. use either

show innodb status
show variables like ‘inno%’;

also what your key buffer size, sort buffer size and query cache setting?
Actually if you look the first half of this article, Baron already provided a good list to look at when innodb not used.

By: Baron Schwartz

Baron Schwartz — Thu, 19 Feb 2009 13:33:05 +0000

That question is better asked on mailing lists, forums, an IRC channel, or addressed by hiring us You’ve admitted that you’ve been hired yourself to provide an answer to the problem…

By: Heiðar Eiríksson

Heiðar Eiríksson — Thu, 19 Feb 2009 11:35:15 +0000

Hello, and thanks for all the great information on this site.

I have a huge problem.
I was recently hired to “fix” a web that Realestate agents use to advertise properties for sale and/or rent.

We are currently using MySQL and MyISAM tables and we keep running out of memory so the server starts to swap and then crashes.

What makes me realy scratch my head is that the database isn’t even that big….the rows in the property table ar only about 15500 at the moment.

In the runtime information i can see that we have about 177.23 k select queries per hour wich accounts for 91.48% of the server load.

There are a bunch of varibales that jump out as not good to be, here are a few:
Innodb_buffer_pool_reads 12
Handler_read_rnd 2,448 k(this increases steady as the server runs)
Handler_read_rnd_next 105 M(this increases steady as the server runs)
Created_tmp_disk_tables 9,208
Select_full_join 1,074(the application is using indexes to join the tables so this puzzles me)
Sort_merge_passes 2,602
Opened_tables 1,500
Table_locks_waited 74

hope this is enough information to help you help me.

The server has 6GB memory running on Linux
and a

Is it the queries that are to blame here or could i optimize the config for the server some….maybe use InnoDB since the database is 3NF and uses alot of selects using joins?

Any tips and help would be immensly helpfull.

Thanks in advance,
Heiðar Eiríksson

By: Shane Bester

Shane Bester — Wed, 18 Feb 2009 12:33:37 +0000

Hi!

I reported this strange memory consumption formula as a bug after I got tired of questions about it:

http://bugs.mysql.com/bug.php?id=35661

By: Baron Schwartz

Baron Schwartz — Mon, 16 Feb 2009 14:37:38 +0000

I doubt that Monty is the right person to ask to fix InnoDB Percona and Oracle are the experts there. You can also hire us to figure out what’s wrong here (I didn’t say that before, because it seems like stating the obvious).

By: Sheeri K. Cabral

Sheeri K. Cabral — Mon, 16 Feb 2009 12:30:05 +0000

Brice,

Monty recently posted that his company does NRE, or Non-Recurring Engineering. This is explained on http://askmonty.org/wiki/index.php/Commercial_offerings as “Provide bug fixes for MySQL that is not fixed in the standard MySQL release from Sun.”

I think it would be a good idea to send him an enquiry.

By: Baron Schwartz

Baron Schwartz — Sat, 14 Feb 2009 16:28:58 +0000

Brice, it surely seems that something weird is going on with InnoDB. I’ve personally built systems where temporary InnoDB tables are created and dropped like mad, and not seen this behavior. You shouldn’t be using that much memory for 300 tables.

By: Brice Figureau

Brice Figureau — Fri, 13 Feb 2009 18:27:02 +0000

@Arjan,
I don’t really know if it’s a memory leak but it didn’t happened when we were running under 4.0. I reported this issue around 5.0.27 to mysql, and they indeed find one memory leak. Unfortunately this didn’t fix my issue.
We’re running with short lived connections, and with a wait timeout of 120s, since that was my first thought (I mean long lived connections from our game servers).

@Baron,
Here is my mem status:

Total memory allocated 8751883484; in additional pool allocated 20926208
Buffer pool size 458752
…

Which mean per your formula:
mysql> select 8751883484 – (458752*16384) – 20926208;
+—————————————-+
| 8751883484 – (458752*16384) – 20926208 |
+—————————————-+
| 1214764508 |
+—————————————-+

That’s 1.2GB for 1.5 week of uptime, ouch.

We have about 300 innodb tables. But about 20 of them are truncated every 5s (not dropped) as they are used to compute online games rankings. Maybe that’s temporary table ending in the dictionnary?

I’m not running a percona build (yet I’d say, I’m in the process of building a debian package out of the latest release).

I tried everything I could to diagnose this, including running with different debug allocator (like the google perftool) , or on top of valgrind. Unfortunately usually the server dies (because there is too much load) before giving much information.
So, yes maybe that’s the dictionnary, because when we were running 4.0 (and much less load) we were a MyISAM only shop.
Thanks,
Brice

By: Baron Schwartz

Baron Schwartz — Fri, 13 Feb 2009 14:03:18 +0000

Bruce,

Look at SHOW INNODB STATUS and see this part:

BUFFER POOL AND MEMORY
———————-
Total memory allocated 17475088; in additional pool allocated 864256
Buffer pool size 512

Multiply the buffer pool size by the page size (or just check my.cnf) and subtract that and the additional pool from the total memory allocated. For example,

mysql> select 17475088 – (512 * 16384) – 864256;
+———————————–+
| 17475088 – (512 * 16384) – 864256 |
+———————————–+
| 8222224 |
+———————————–+

Do you have a lot left over? Sometimes the difference is gigabytes. It’s usually the data dictionary — do you have tens of thousands of tables or more?

If that’s not it, are you running a Percona patched build? There was actually a memory leak in one of our patches a few releases back! I can ask Vadim for the details… I forgot it already.