Michael

This uses FileSort (external sort) which is fatally slow for millions of items we’re looking at.

Regarding your comment about Full Text Search this is too simplified understanding The process you’re describing is stemming – which can be adjusted. In this case we do not use stemming.

Is it just adding some extra columns to paths? presumably not

Full text search breaks down the words into lexemes (as it does for searches queries) so that animation, animator and animated will all be returned for searches for animate.

I’m still under the impression that this is not the behaviour you are exploiting ergo there must be a better way of doing it.

Your example does not solve the main problem I’m mentioning – sorting together with prefix/range index.

Regarding LIKE – you can have it case sensitive or case insensitive depending on collation you’re using.

What a lame way to sql escape.

Here’s my postgresql table layout though it looks fairly SQL standard to me.

CREATE TABLE domains (
domain VARCHAR NOT NULL PRIMARY KEY
);
CREATE TABLE subdomains (
sub VARCHAR NOT NULL,
domain REFERENCES domains ON CASCADE DELETE,
PRIMARY KEY (sub, domain)
);
CREATE TABLE paths (
proto VARCHAR NOT NULL;
path VARCHAR NOT NULL,
port INT,
domain varchar NOT NULL REFERENCES domains ON CASCADE DELETE,
sub varchar REFERENCES subdomains ON CASCADE DELETE,
);
CREATE INDEX paths(sub, domain, port, path);

SELECT proto || “://” ||
coalesce(sub || “.”, “”) || domain ||
coalesce(“:” || port, “”)|| path
FROM paths
WHERE domain=”mysql.com”
AND path LIKE “/downloads/%”;

fun with I/LIKE – some web servers are case insensitive!

I think Sphinx SE Is a good way for MySQL Users to start using Sphinx not getting out of MySQL environment.

In our projects we typically use native Sphinx API instead which allows to keep stock MySQL version and is more transparent in terms of understanding performance properties.

It is actually quite easy – get list of row IDs from sphinx, retrieve data from MySQL, use sphinx to do filtering, ordering and grouping.

We used to use similar approach. You’re right in this case MySQL uses index to retrieve data, the problem however for some URLs there would be millions of of links which have to be sorted with “filesort” which is too bad.

If the goal would be to show top 10 any links this approach would work just fine