February 4, 2008

Finding out largest tables on MySQL Server

Posted by peter

Finding largest tables on MySQL instance is no brainier in MySQL 5.0+ thanks to Information Schema but I still wanted to post little query I use for the purpose so I can easily find it later, plus it is quite handy in a way it presents information:

PLAIN TEXT
SQL:
  1. mysql> SELECT concat(table_schema,'.',table_name),concat(round(table_rows/1000000,2),'M') rows,concat(round(data_length/(1024*1024*1024),2),'G') DATA,concat(round(index_length/(1024*1024*1024),2),'G') idx,concat(round((data_length+index_length)/(1024*1024*1024),2),'G') total_size,round(index_length/data_length,2) idxfrac FROM information_schema.TABLES ORDER BY data_length+index_length DESC LIMIT 10;
  2. +-------------------------------------+--------+--------+--------+------------+---------+
  3. | concat(table_schema,'.',table_name) | rows   | DATA   | idx    | total_size | idxfrac |
  4. +-------------------------------------+--------+--------+--------+------------+---------+
  5. | art87.link_out87                    | 37.25M | 14.83G | 14.17G | 29.00G     |    0.96 |
  6. | art87.article87                     | 12.67M | 15.83G | 4.79G  | 20.62G     |    0.30 |
  7. | art116.article116                   | 10.49M | 12.52G | 3.65G  | 16.18G     |    0.29 |
  8. | art84.article84                     | 10.10M | 10.11G | 3.59G  | 13.70G     |    0.35 |
  9. | art104.link_out104                  | 23.66M | 6.63G  | 6.55G  | 13.18G     |    0.99 |
  10. | art118.article118                   | 7.06M  | 10.49G | 2.68G  | 13.17G     |    0.26 |
  11. | art106.article106                   | 9.86M  | 10.19G | 2.76G  | 12.95G     |    0.27 |
  12. | art85.article85                     | 6.20M  | 9.82G  | 2.51G  | 12.33G     |    0.26 |
  13. | art91.article91                     | 8.66M  | 9.17G  | 2.66G  | 11.83G     |    0.29 |
  14. | art94.article94                     | 5.21M  | 10.10G | 1.69G  | 11.79G     |    0.17 |
  15. +-------------------------------------+--------+--------+--------+------------+---------+
  16. 10 rows IN SET (2 min 29.19 sec)

I do some converting and rounding to see number of rows in millions and data and index size in GB so I can save on counting zeros.
The last column shows how much does the index take compared to the data which is mainly for informational purposes but for MyISAM can also help you to size your key buffer compared to operating system cache.

I also use it to see which tables may be worth to review in terms of indexes. Large index size compared to data size often indicates there is a lot of indexes (so it is well possible there are some duplicates, redundant or simply unused indexes among them) or may be there is long primary key with Innodb tables. Of course it also could be perfectly fine tables but it is worth to look.

Changing the query a bit to look for different sorting order or extra data - such as average row length you can learn quite a lot about your schema this way.

It is also worth to note queries on information_schema can be rather slow if you have a lot of large tables. On this instance it took 2.5 minutes to run for 450 tables.

UPDATE: To make things easier I've added INFORMATION_SCHEMA to the query so it works whatever database you have active. It does not work with MySQL before 5.0 still of course :)

Posted by peter @ 7:46 am :: tips  
Print This Post Print This Post book mark Finding out largest tables on MySQL Server in del.icio.usdel.icio.us :: submit Finding out largest tables on MySQL Server to digg.comdigg  
Comment RSS :: Trackback URI  
Related posts: :Researching your MySQL table sizes::Finding the Percona Performance Conference::Feature Idea: Finding columns which query needs to access:
 

21 Comments »

  1. 1. Unomi

    I’m sorry, but where does ‘TABLES’ come from? Do I miss something here? Is it something I’m supposed to know, but is not mentioned in the article?

    I run this query against the ‘mysql’ database on 5.0.51, but mysql.TABLES cannot be found. Where should I look for the missing link?

    - Unomi -

    Comment :: February 4, 2008 @ 8:53 am

  2. 2. Matthew Montgomery

    Unomi,

    This is a query against information_schema.TABLES. The information_schema database was added at v5.0

    Matt,

    Comment :: February 4, 2008 @ 9:19 am

  3. 3. Frank Mash

    Unomi, you need to use information_schema:

    mysql> use information_schema;

    Comment :: February 4, 2008 @ 9:21 am

  4. 4. Jay Pipes

    Great stuff, Peter! I encourage you to add this to the Forge snippets repository! :)

    Cheers,

    Jay

    Comment :: February 4, 2008 @ 11:04 am

  5. 5. Gabriel Menini

    Same error here…

    Comment :: February 4, 2008 @ 12:19 pm

  6. 6. David Linsin

    Awesome Peter, thanks!

    Comment :: February 4, 2008 @ 10:44 pm

  7. 7. Gabriel Menini

    Frank, thanks for the tip :-) It’s working now.

    Comment :: February 5, 2008 @ 5:46 am

  8. 8. Unomi

    Thanks all for the tip. It’s new for me, but I wanted to have this query running… Everyday a new day to learn something!

    - Unomi -

    Comment :: February 6, 2008 @ 12:46 am

  9. 9. Radek

    Example works perfectly.

    Great website and great ideas, keep working dude.

    Comment :: February 6, 2008 @ 3:36 pm

  10. 10. gigiduru

    Peter,

    Why exactly is it taking 2 min 29.19 sec to extract that data?! I thought it’s readily available in the dictionary. And I don’t believe it’s taking a lot of processing power to transform, concatenate and order the results.
    Also, this is a thing MySQL AB should work on to fix it (among other several thousands of not-fixed-yet bugs).

    Comment :: February 12, 2008 @ 12:07 pm

  11. 11. peter

    There is no readily available data dictionary in MySQL. The information schema actually have to scan through all database and gets stats from each of the tables and put them into temporary table to have them available to you. Sometimes it may restrict the scan ie if you look for given table but in this case it has to scan through all tables.

    Comment :: February 12, 2008 @ 12:20 pm

  12. 12. Chris

    Thanks Peter, this was a great help.

    Comment :: February 13, 2008 @ 9:10 am

  13. 13. Finding out largest tables on MySQL Server « Purab’s Programmer Blog

    [...] purpose so I can easily find it later, plus it is quite handy in a way it presents information: PLAIN TEXT [...]

    Pingback :: February 15, 2008 @ 12:23 am

  14. 14. The Linux Index » Stephan Hermann: How to determine the largest table in your MySQL database?

    [...] "Finding out largest tables on MySQL server" [...]

    Pingback :: February 28, 2008 @ 11:34 am

  15. 15. Yafei Qin

    I have the same question when it takes more than half minute during check my all database, as gigiduru mentioned.
    Thanks peter. :)

    btw, a tips:
    If you want to show tables only in a certain database, add a WHERE clause in the SQL
    WHERE table_schema = ‘db_name’

    Comment :: March 4, 2008 @ 7:35 pm

  16. 16. Researching your MySQL table sizes | MySQL Performance Blog

    [...] posted a simple INFORMATION_SCHEMA query to find largest tables last month and it got a good response. [...]

    Pingback :: March 17, 2008 @ 7:28 pm

  17. 17. Zeck’s Blog » Archive du blog » MySQL, quelques requĂȘtes pour l’administration…

    [...] Finding out largest tables on MySQL Server [...]

    Pingback :: August 8, 2008 @ 3:05 am

  18. 18. vladislav

    Nice. Thanks for your time and effort. Open Source is cool.

    Comment :: October 6, 2008 @ 7:49 am

  19. 19. lvermilion

    Nice post!! The only issue I have is it is not the same result from “select count(*) from “. For that matter the show table status is not the same either. The true size of the table is increasing and I can not seem to find records getting deleted anywhere. Can you explain why this is?

    Here is an example.

    mysql> select count(*) from table;
    +———-+
    | count(*) |
    +———-+
    | 14828558 |
    +———-+
    1 row in set (39.94 sec)

    #########
    Now to make it interesting, I will to the show table status and your SQL statement and get different results from the count(*). (Note the “show table status” results match the output of your SQL statement, if I could execute them at identical seconds).
    #########

    mysql> SELECT concat(table_schema,’.',table_name),concat(round(table_rows/1000000,2),’M') rows,concat(round(data_length/(1024*1024*1024),2),’G') DATA,concat(round(index_length/(1024*1024*1024),2),’G') idx,concat(round((data_length+index_length)/(1024*1024*1024),2),’G') total_size,round(index_length/data_length,2) idxfrac FROM information_schema.TABLES ORDER BY data_length+index_length DESC LIMIT 10;
    +———————————————-+——–+——-+——-+————+———+
    | concat(table_schema,’.',table_name) | rows | DATA | idx | total_size | idxfrac |
    +———————————————-+——–+——-+——-+————+———+
    | database.table | 14.73M | 2.08G | 0.00G | 2.08G | 0.00 |
    +———————————————-+——–+——-+——-+————+———+
    10 rows in set (0.15 sec)

    mysql> show table status like ‘table’ \G
    *************************** 1. row ***************************
    Name: table
    Engine: InnoDB
    Version: 10
    Row_format: Compact
    Rows: 14898977
    Avg_row_length: 150
    Data_length: 2237661184
    Max_data_length: 0
    Index_length: 0
    Data_free: 0
    Auto_increment: NULL
    Create_time: 2008-10-01 09:38:00
    Update_time: NULL
    Check_time: NULL
    Collation: latin1_swedish_ci
    Checksum: NULL
    Create_options:
    Comment: InnoDB free: 4021248 kB
    1 row in set (0.06 sec)

    ########
    I have ran your query and the show table status query yet again and get new values that are not consistent with “select count(*) from “.
    ########

    +———————————————-+——–+——-+——-+————+———+
    | concat(table_schema,’.',table_name) | rows | DATA | idx | total_size | idxfrac |
    +———————————————-+——–+——-+——-+————+———+
    | database.table | 15.00M | 2.08G | 0.00G | 2.08G | 0.00 |
    +———————————————-+——–+——-+——-+————+———+

    mysql> show table status like ‘table’ \G
    *************************** 1. row ***************************
    Name: table
    Engine: InnoDB
    Version: 10
    Row_format: Compact
    Rows: 15120698
    Avg_row_length: 147
    Data_length: 2237661184
    Max_data_length: 0
    Index_length: 0
    Data_free: 0
    Auto_increment: NULL
    Create_time: 2008-10-01 09:38:00
    Update_time: NULL
    Check_time: NULL
    Collation: latin1_swedish_ci
    Checksum: NULL
    Create_options:
    Comment: InnoDB free: 4021248 kB
    1 row in set (0.02 sec)

    mysql> select count(*) from table;
    +———-+
    | count(*) |
    +———-+
    | 14828945 |
    +———-+
    1 row in set (44.33 sec)

    Comment :: October 6, 2008 @ 12:10 pm

  20. 20. lvermilion

    If I check mysql reference show table status seems to be very inaccurate if we take it for the number of rows, but the datalength seems to be accurate. Is your SQL statement keying off the same values that show table status is?

    Rows

    The number of rows. Some storage engines, such as MyISAM, store the exact count. For other storage engines, such as InnoDB, this value is an approximation, and may vary from the actual value by as much as 40 to 50%. In such cases, use SELECT COUNT(*) to obtain an accurate count.

    Comment :: October 6, 2008 @ 12:16 pm

  21. 21. Gheek.net » MySQL add an index to a table.

    [...] Referenced from: mysqlperformanceblog.com Finding largest tables on MySQL instance is no brainier in MySQL 5.0+ thanks to Information Schema but I still wanted to post little query I use for the purpose so I can easily find it later, plus it is quite handy in a way it presents information: SQL: mysql> SELECT concat(table_schema,’.',table_name),concat(round(table_rows/1000000,2),’M') rows,concat(round(data_length/(1024*1024*1024),2),’G') DATA,concat(round(index_length/(1024*1024*1024),2),’G') idx,concat(round((data_length+index_length)/(1024*1024*1024),2),’G') total_size,round(index_length/data_length,2) idxfrac FROM information_schema.TABLES ORDER BY data_length+index_length DESC LIMIT 10; +————————————-+——–+——–+——–+————+———+ | concat(table_schema,’.',table_name) | rows | DATA | idx | total_size | idxfrac | +————————————-+——–+——–+——–+————+———+ | art87.link_out87 | 37.25M | 14.83G | 14.17G | 29.00G | 0.96 | | art87.article87 | 12.67M | 15.83G | 4.79G | 20.62G | 0.30 | | art116.article116 | 10.49M | 12.52G | 3.65G | 16.18G | 0.29 | | art84.article84 | 10.10M | 10.11G | 3.59G | 13.70G | 0.35 | | art104.link_out104 | 23.66M | 6.63G | 6.55G | 13.18G | 0.99 | | art118.article118 | 7.06M | 10.49G | 2.68G | 13.17G | 0.26 | | art106.article106 | 9.86M | 10.19G | 2.76G | 12.95G | 0.27 | | art85.article85 | 6.20M | 9.82G | 2.51G | 12.33G | 0.26 | | art91.article91 | 8.66M | 9.17G | 2.66G | 11.83G | 0.29 | | art94.article94 | 5.21M | 10.10G | 1.69G | 11.79G | 0.17 | +————————————-+——–+——–+——–+————+———+ 10 rows IN SET (2 min 29.19 sec) [...]

    Pingback :: October 14, 2008 @ 11:30 am

 



Subscribe without commenting