SELECT count(*) FROM table_a LIMIT 1000;

So if the table has more than 1k rows, it should just return and say “1000″. What i’m trying to avoid is have the entire table scanned, in particular if it has millions of rows. Basically the question i want to ask is: “Does table X have at least N rows?”.

Yes, I was talking about this: “Typically we do a count first to create all the pagination stuff. Current page, total pages, etc.” - you have to create a counter for pagination information.

Thank you for your solution, maybe I will try it on other projects. Right now I’m fine with solution posted above by me.

Example of it in action here. http://www.pinkbike.com/photo/list/?date=all&page=6
You can page to page 100,000 (24 per page, so the offset is 2.4 million) without and slow down. If this was done in one call it literally takes seconds to generate offsets of 2,000,000

Yes, you can order by in the first query by id no problem. The id is indexed so no penalty there. Also your result set can be ordered by also but that is fast as you only have 24 items, or however many you show on the page.

Cheers

Have you tried?

I’m Joining multiple tables, so the join must be done in the second query..

Unfortunately, with your solution I have to use 3 queries:
- SELECT COUNT(*) .. (for total rows that meets the criteria blabla)
- SELECT id … LIMIT x,y
- SELECT id, name .. IN (x1,x2…x20)

Anyway.. thank you for the tweak!

Example (category,id index)
select photo.id, photo.description from photo where category=1 limit 30000000,20
Query time: 4 seconds

To speed things up we split the call into 2 queries

select photo.id from photo where category=1 limit 30000000,20
Query time: 0.030 (this just uses the index so no traversing the data table)
Select photo.id, photo.description from photo where photo.id IN (x1,x2…x20)
The x1 - x20 are the results the first query returns
Query time: 0.020

So with a simple change like this we sped things up by a couple of orders of magnitude and allow our users to use large pagination.

I assume it has something to do that the the first query only uses the index to get the ids and does not have to jump around getting a larger data set.

Cheers.

Thanks.

SELECT
*
FROM
message

LEFT JOIN
accounts a ON message_account = a.id

LEFT JOIN
opportunities o ON message_project = o.id

LEFT JOIN
leads l1 ON (UCASE(message_from) LIKE CONCAT(\’%\’, UCASE(l1.email1), \’%\’)
AND l1.email1 \’\') OR (UCASE(message_from) LIKE CONCAT(\’%\’,
UCASE(l1.email2), \’%\’) AND l1.email2 \’\')

LEFT JOIN
leads l2 ON (UCASE(message_to) LIKE CONCAT(\’%\’, UCASE(l2.email1), \’%\’)
AND l2.email1 \’\') OR (UCASE(message_to) LIKE CONCAT(\’%\’,
UCASE(l2.email2), \’%\’) AND l2.email2 \’\')

WHERE
message_visibility
AND message_mailbox = UCASE(\’user\’)
AND message_visibility 1

ORDERBY
message_ocode DESC

SELECT *
FROM forum_posts AS pa
LEFT JOIN forum_posts_text AS pb ON pa.post_id = pb.post_id
LEFT JOIN forum_users ON user_id = post_poster
WHERE post_topic_id = ‘224′
AND pa.post_id >= (
SELECT post_id
FROM forum_posts
WHERE post_topic_id = ‘224′
ORDER BY post_id ASC
LIMIT 6075 , 1)
ORDER BY pa.post_id ASC
LIMIT 15

SELECT *
FROM forum_posts AS pa
LEFT JOIN forum_posts_text AS pb ON pa.post_id = pb.post_id
LEFT JOIN forum_users ON user_id = post_poster
WHERE post_topic_id = ‘450′
ORDER BY pa.post_id ASC
LIMIT 5475 , 15

versus

SELECT *
FROM forum_posts AS pa
LEFT JOIN forum_posts_text AS pb ON pa.post_id = pb.post_id
LEFT JOIN forum_users ON user_id = post_poster
WHERE post_topic_id = ‘224′
AND pa.post_id >= (
SELECT MAX( post_id )
FROM (
SELECT post_id
FROM forum_posts
WHERE post_topic_id = ‘224′
ORDER BY post_id ASC
LIMIT 6075 , 1
) AS tmp )
ORDER BY pa.post_id ASC
LIMIT 15

~ 0.09 sec versus ~ 0.004 sec

Tables with posts: 2×292,000+ rows (25MB + 89MB)
Topic: 6092 rows (rows that meet condition WHERE post_topic_id = ‘224′)

Source: http://www.easemarry.com/blog/mysql-limit-optimization/

If you found something better, I would be glad to hear.

$sqls=”select id,state from states”;
$rss=mysql_query($sqls) or die(mysql_error());
$num_rowss=mysql_num_rows($rss);

$count=1;

while($num_rowss && $datas=mysql_fetch_assoc($rss))
{

$state_id=$datas['id'];
$state=$datas['state'];
$sqlc=”select * from companies,package where comp_status=1 and comp_account_site=0 and comp_balance >0 and comp_mem_pac=pac_id order by pac_amount DESC, rand()”;

$rsc=mysql_query($sqlc) or die(mysql_error());
$num_rowsc=mysql_num_rows($rsc);

$checkonlyonetime=1; // header
while($num_rowsc && $datac=mysql_fetch_assoc($rsc)) // loop of company
{

$comp_id=$datac['comp_id'];

$comp_name=$datac['comp_name'];
$comp_mapped_city=$datac['comp_mapped_city'];
@$comp_mapped_city=explode(”,”,trim($comp_mapped_city,”,”));

}
}

This is where I found very useful matters about optimization which I have been looking for so long.

Thanks again.

Mathu Mou