Comments on: Computing 95 percentile in MySQL

By: Baron Schwartz

Baron Schwartz — Mon, 08 Dec 2008 16:20:09 +0000

Histograms are nice, I agree. We’ve built histograms into Maatkit’s new log analysis tool, mk-log-parser. We are also going to add an indication of time-clustering, e.g. “this query always runs at 4AM on Tuesday”.

I might write up a post on histograms — with LOG() it is actually not too hard to do, though it’s not terribly efficient (filesort/temp table).

By: peter

peter — Sun, 30 Nov 2008 02:20:32 +0000

Dale,

Histogram is good though takes more work to get in SQL plus it is harder to define the goals. It is much easier to say 95 percentile should be within a second and 99 percentile within 3 seconds than define goals in terms of histogram.

By: Dale Lancaster

Dale Lancaster — Sat, 29 Nov 2008 23:42:53 +0000

Thanks for the article. For me having a histogram of access/load times would be more meaningful. I am not sure what a 99 or 95 percentile ranking would mean to me. If anything it would lead me to generate more statistics and the one I would found most useful is a histogram of the counts of the load times. This would require normalizing the data a bit such that only the first couple decimal places of precision are used, otherwise you get a lot of noise.

To generate a histogram with SQL I usually just use the group by of a count on the load times and you are good to go. So in your example, if you did the group by with order by on the load time, I would be most interested in how often the really large load times occur.

Thanks.