Comments on: Opening Tables scalability

By: peter

peter — Thu, 17 Apr 2008 16:31:56 +0000

MySQL only opens tables once you access them.

By: art

art — Thu, 17 Apr 2008 11:13:37 +0000

Does mysql open all innodb tables at startup or only with query on that particular table?

By: Sysadm

Sysadm — Tue, 20 Mar 2007 19:29:05 +0000

We have forum hosting community - around 3 500 000 myisam tables per each server. Each forum gets it’s own database (~90 tables in one database) . We have similar issue - when server is overloaded, status of all threads in “show processlist” is “opening tables”. We have many php and system tweaks but this problem seems to be hard to resolve without MySQL source code rewrite -we have not enought knowledge to do it. It’s MySQL 4.0.

We have tried to increase table_cache but it doesn’t make any sense with such many useable tables - best performance setting for us is table_cache=0.

Now it’s hosted on Linux 2.6 & XFS which supports more than 32K subdirectories in one directory. Reiserfs supports too, but it’s terribly unstable within such load and brokes it’s journal itself very often, and reiserfs is not visible faster in this than XFS.

I’m afraid of putting such many tables into one database - this cause worse performance problems - like hosting >10 000 000 files in one directory (3 files per table)…

By: peter

peter — Sun, 11 Mar 2007 12:49:23 +0000

This actually was always curious for me why Wordpress MU was not fixed to operate with smaller amount of tables rather than create single table set for each users.

You can blame MySQL for certain performance problems but generally dealing with a lot of files is slower with most of file systems.

By: Hakan

Hakan — Sun, 11 Mar 2007 10:25:20 +0000

Wordpress.com hosts a free blog system running WordPress Mu (multiuser wordpress) and it creates a new set of tables for every new blog.

/Hakan

By: peter

peter — Thu, 08 Mar 2007 08:36:35 +0000

Steven,

Alexey is saying MySQL Privilege system may be slow with so huge number of databases and a lot of grants on database level.

Regarding Matt, as I understand Matt is using standard software as a base which has its own set of tables for each user. If you have something custom having so many tables is of course not good idea.

By: Steven Roussey

Steven Roussey — Thu, 08 Mar 2007 00:05:57 +0000

The 32000 limit is in ext2 and ext3 (still not fixed for ext4, although there is an old patch to make it 64000). If you happen to be using ext as a file system with a lot of either databases or file systems, you might try adding dir_index and then do a tune. Don’t know what the speed up is, your mileage may vary.

Matt: why 5 million tables? We host 500,000 forums and don’t give each one its own set of tables, it was just too much work and too slow and complex.

By: peter

peter — Thu, 28 Dec 2006 12:46:40 +0000

Alexey,

OK This is different story when privilege check system comes into play. I guess it was not optimized well to handle such large amount of tables. I remember seeing quite a few of list scans when I worked with code few years ago.
I would suggest to post it as a bug and hope it would be fixed sometime.

By: Matt

Matt — Sun, 24 Dec 2006 10:03:40 +0000

If anyone is interested in improving the way MySQL deals with a ton of table we (WordPress.com) would happily sponsor that development. We have about 5 million tables today and it’s going up constantly. I’m reachable on my site.

By: Alexey

Alexey — Fri, 22 Dec 2006 20:49:04 +0000

Here’s a piece of profiling data from live system, running lot of DBs without too much load (about 80 qps, select-heavy with pretty good hit rate):
root@titanic [/var/lib/mysql]# ls -la|wc -l
24547
root@titanic [/var/lib/mysql]# opreport -l *mysqld|head
warning: /oprofile could not be found.
CPU: AMD64 processors, speed 2394.07 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0×00 (No unit mask) count 100000
samples % image name symbol name
10174899 26.6171 libc-2.3.4.so strcmp
5460631 14.2848 mysqld acl_getroot(THD*, user_resources*, char const*, unsigned int)
2592977 6.7831 libc-2.3.4.so memcpy
1956617 5.1184 vmlinux-2.6.17.7 copy_user_generic_c
1797812 4.7030 mysqld Query_cache::insert_into_free_memory_list(Query_cache_block*)
1685928 4.4103 mysqld my_lengthsp_8bit
1066380 2.7896 mysqld anonymous symbol from section .plt

First two functions are privilege-system related (strcmp is used for comparing logins, filenames etc., mysql uses it’s own functions for matching data). As you can see those two functions alone consume more than 40% of cpu time. That’s after fixing show databases problem, with show databases the number would be more like 95%.