The reason for why I created TBGsearch was not to build a complete standalone search engine solution but to find a replacement of mysql fulltext for the forum at http://www.tbg.nu

In my case the problem with mysql fulltext were that my searches performed:

* Using boolean search.
* No stopword-list.
* Sorting hits by date and not “relevance” since relevance were irrelevant in this case

All the three reasons above made mysql fulltext to virtually puke when a client performed a search which contained several commonly existing word.

I assume the reason for that is that the mysql fulltext internally will first find all hits for each word (since it was boolean search), then sort the result on date and finally return top 200 matches (instead of having some optimization to loop one hit at a time until 200 hits is collected or similar to that). Ending up taking sometimes 10 minutes or more to complete a search…

So I started to look for a fix or a replacement and quickly found sphinx which I created a perl api for (since I’m using perl on my site). However sphinx had some limitations such as no support for wildcards, no liveupdates and when a new sphinx version were released the old perl api was no longer compatible.

So instead of putting time into updating the perl api I tried to see how large the difference would be if I created a search engine written only in perl with mysql as backend for the storage and this ended up in TBGseach which I have released as open-source at http://www.tbg.nu/tbgsearch (for download http://www.tbg.nu/tbgsearch/tbgsearch.zip).

In case someone has some tips or other questions regarding TBGsearch you can email me at the address which is written in http://www.tbg.nu/tbgsearch/readme.txt

I can’t agree with you on final conclusion. First you have to identify your performance goals and than check solutions. For many data sizes or search requirements MySQL FullText search performance is unfortunately showstopper.

There are some examples and benchmarks in presentation

For tiny data size, ie search for your personal DVD collection MySQL FullText search should be used of course as it is simplest to use.

You also can check sphinx which goes between Lucene and MySQL FullText Search in features and ease of use but which is the fastest.

Note: It might sounds as I’m too negative to MySQL Full Text Search but just have seen too many people being stuck trying to scale it when their data size grew.

I’ve worked with both Lucene and MySQL full-text search and both are fairly decent. MySQL really doesn’t scale as much as Lucene but it’s really easy to implement and support. Much better to just use MySQL if you’re already a MySQL shop then to have to deal with Lucene.

Kevin