Could that difference have been caused by a large part of that query being cached the second time?

I had a slow query that involved pagination, so I investigated after reading your post on the matter. The result of my investigation was very interesting. Basically, I had a query that was taking about 3 seconds to execute. The SELECT portion of the query was specifying which columns to fetch (i.e. SELECT table.col2, table.col2, … table.col9). When I simply changed this to “SELECT table.*” the query time dropped to 0.01 sec.! So, in the end, it did not appear that the slowness was a result of the pagination at all (the query uses ORDER BY, GROUP BY, WHERE and LIMIT and orders on a non-indexed field.) Using EXPLAIN did not show any different in the query optimization path for the two forms of query either. MySQL bug?

Sam

My solution that works very well and has good performance and does NOT remove functionality as this article suggests is:

I’ll explain how I do it for the forum:
———————————————–
1) I have a statistics table that is updated by triggers on inserts:
statistics_table(serviceid, count, lastinsert):
+——–+———+—————-+
| SERVICE| COUNTER | LASTINSERT |
|——–|———|—————-|
| forum | 10020202| 2008-10-03…. |
| users | 10002| 2008-09-13…. |

(I use InnoDB so getting count(*) is too slow)

2) Per forum category I added the same statistics per item in the forum-category table. Again, this table is updated with the same trigger that updates the statistics_table

+———+———+—————-+
| CATEGORY| COUNTER | LASTINSERT |
|———|———|—————-|
| GENERAL | 10001| 2008-10-03…. |
| TOPIC1 | 1002| 2008-09-13…. |
| TOPIC2 | 4002| 2008-09-13…. |

Then, based on the query’s where-statements I get the total rows available if its an empty where statement or the counter from the forum-category table if filtered on a category. If its anything else, like free text search or filtered on a user etc. then I fall back to the SQL_COUNT_FOUND_ROWS method.
The tables are so small that they are always cached. I wanted to use in-memory engine, but for some reason I got empty results when querying the forum-category table when in memory.

Making paging links is then easy using (total rows)/(rows per page)

David

mysql_connect( “host”, “user”, “passwd” );
mysql_select_db( “db” );
$res = mysql_query( “EXPLAIN SELECT * FROM users” );
if( $res )
{
$estimate = 1;
while( $row = mysql_fetch_array($res, MYSQL_ASSOC) )
{
$estimate *= $row['rows'];
}
echo “Estimate rows: $estimate.\n”;
}
else
{
echo “SQL error.\n”;
}