Comments on: Just do the math!

By: Apachez

Apachez — Sat, 11 Jul 2009 22:27:24 +0000

For example number of rows have been optimized in myisam to be a single value so table scan isnt needed (compared to innodb) in order to find out how many rows the table contains.

Have there been any talks of optimizing similar stuff for mysql 6.0 or such (like a storage independent aggregation)?

If we take the example in this blogentry it would be nice if mysql would have done the aggregation on its own in the index regarding the count of each unique page entry so the select in the example would go in msecs instead of minutes or hours. So I as a user of a mysql database only need to care to throw in the data (like from the apache log in realtime). Compared to today where I most likely would need to take care of the aggregation on my own with 2 inserts per log line instead of just 1 (one to the large log db and one to the aggregated db).

If im not mistaken mssql does similar stuff since a while back.

By: Brian

Brian — Wed, 08 Jul 2009 15:07:01 +0000

You would be surprised how controversial “doing the math” is. To most the database is supposed to magically compensate for whatever you throw at it regardless of how its configured or your data model is designed. Saying otherwise is well, “being mean”.

By: peter

peter — Wed, 08 Jul 2009 00:06:17 +0000

Pat,

There are always some surprises. If you got 10% more data and queries started taking 4x there is a good chance something is wrong. May be the execution plan has changed ?
It is also however possible to have large drops in performance because of working set. Consider for example if table has uniform random access when it fits in memory (100% hit ratio) vs 10% miss ratio if it grows 10%. You can access data in memory at rate of 500.000 rows/sec while if it is the disk it is 200 rows/sec – with such numbers 10% miss rate will cause 500K -> 2K rows/sec which is 250 _times_ performance loss.

By: pat

pat — Tue, 07 Jul 2009 13:43:33 +0000

There are some non linear points to worry about too though aren’t there? I’d expect to see non linear timing changes when my data:

1) grew large enough that it didn’t fit in buffer cache anymore
2) grew large enough it didn’t fit in sort buffers anymore and had to spill to disk

For that vast “middle” size of data you can get the more pernicious performance problems of “it was fine last week, and its only 10% larger this week, but the queries are all taking 4x as long”.