Comments on: Opening Tables scalability

By: Derek Organ

Derek Organ — Tue, 27 Oct 2009 15:42:09 +0000

We work on the basis of a different database for each new company account. The problem I’m starting to have with 5000 databases is the show databases and use database commands are taking a long time to run and are showing up in my slow-queries log. Are there any tips for making this go faster?

By: My “hot” list for next InnoDB features | MySQL Performance Blog

My “hot” list for next InnoDB features | MySQL Performance Blog — Tue, 31 Mar 2009 21:48:29 +0000

[...] bad at start time, when InnoDB takes probes during opening table, as it is slow operation. See also http://www.mysqlperformanceblog.com/2006/11/21/opening-tables-scalability/. Partially it can be fixed by recent patches by enabling / disabling probes and changing count of [...]

By: peter

peter — Thu, 17 Apr 2008 16:31:56 +0000

MySQL only opens tables once you access them.

By: art

art — Thu, 17 Apr 2008 11:13:37 +0000

Does mysql open all innodb tables at startup or only with query on that particular table?

By: Sysadm

Sysadm — Tue, 20 Mar 2007 19:29:05 +0000

We have forum hosting community – around 3 500 000 myisam tables per each server. Each forum gets it’s own database (~90 tables in one database) . We have similar issue – when server is overloaded, status of all threads in “show processlist” is “opening tables”. We have many php and system tweaks but this problem seems to be hard to resolve without MySQL source code rewrite -we have not enought knowledge to do it. It’s MySQL 4.0.

We have tried to increase table_cache but it doesn’t make any sense with such many useable tables – best performance setting for us is table_cache=0.

Now it’s hosted on Linux 2.6 & XFS which supports more than 32K subdirectories in one directory. Reiserfs supports too, but it’s terribly unstable within such load and brokes it’s journal itself very often, and reiserfs is not visible faster in this than XFS.

I’m afraid of putting such many tables into one database – this cause worse performance problems – like hosting >10 000 000 files in one directory (3 files per table)…

By: peter

peter — Sun, 11 Mar 2007 12:49:23 +0000

This actually was always curious for me why Wordpress MU was not fixed to operate with smaller amount of tables rather than create single table set for each users.

You can blame MySQL for certain performance problems but generally dealing with a lot of files is slower with most of file systems.

By: Hakan

Hakan — Sun, 11 Mar 2007 10:25:20 +0000

Wordpress.com hosts a free blog system running WordPress Mu (multiuser wordpress) and it creates a new set of tables for every new blog.

/Hakan

By: peter

peter — Thu, 08 Mar 2007 08:36:35 +0000

Steven,

Alexey is saying MySQL Privilege system may be slow with so huge number of databases and a lot of grants on database level.

Regarding Matt, as I understand Matt is using standard software as a base which has its own set of tables for each user. If you have something custom having so many tables is of course not good idea.

By: Steven Roussey

Steven Roussey — Thu, 08 Mar 2007 00:05:57 +0000

The 32000 limit is in ext2 and ext3 (still not fixed for ext4, although there is an old patch to make it 64000). If you happen to be using ext as a file system with a lot of either databases or file systems, you might try adding dir_index and then do a tune. Don’t know what the speed up is, your mileage may vary.

Matt: why 5 million tables? We host 500,000 forums and don’t give each one its own set of tables, it was just too much work and too slow and complex.

By: peter

peter — Thu, 28 Dec 2006 12:46:40 +0000

Alexey,

OK This is different story when privilege check system comes into play. I guess it was not optimized well to handle such large amount of tables. I remember seeing quite a few of list scans when I worked with code few years ago.
I would suggest to post it as a bug and hope it would be fixed sometime.