It looks like you have 2G file size limit somethere. MySQL does not have 2G limit – even with small data_pointer_size for MyISAM tables you will get at last 4G

So I’d look at OS settings, may be quota.

Is that information publically available for download?

Thanks in advance and keep up the great work!

Thanks in advance.

Indeed, understanding the implications of a DBMS architecture on the application requirements is one of the hardest DBA tasks. The TPC-H and similar applications are meant to aid in this process, albeit a little. Ideally, a user does not require an extensive DBA course to handle reasonably sized database and its application.

The development in the area of column-stores and main-memory databases is moving at great pace.
Some throw hardware at the problem (Kickfire), others exploit different ways of software layering (Vertica, MonetDB). From an software architecture point of view understanding the pro-s and con-s are highly desirable and benchmark comparisons are a step into this direction.

From your comments on april 22, should i conclude that you don’t consider MySQL the premiere choice for datawarehouses?
If it is, did the performance team produce better results on the MySQL versions as presented by Peter at the beginning of this thread, including those beyond the 100GB. What are the tricks and their impact?
I am eager to learn those from the insiders, because the performance of MySQL does not stand out against the fullscale (OLTP +DW) version of MonetDB. Any reference to independent and public available numbers, albeit non-official, are appreciated.

regards, and keep up the good work on database kernels
Martin

ps http://monetdb.cwi.nl/projects/monetdb//SQL/Benchmark/TPCH/ for an indication. Numbers are confirmed recently, trends are the same.

It would be great to know which MySQL settings you used for the 10GB test and 100GB test. What about posting your my.cnf?

Thank you,

-Noah

First this is appliance with 64G of Memory and it uses certain compression for the data which means there was quite good data fit in memory compared to 16G of memory the Dell box had.
Furthermore Kickfire uses column store which reduces IO needed dramatically in case of large scans. It however comes at cost of accessing multiple columns becomes more expensive.

As I mentioned Kickfire indeed handles some kind of queries just great but I would not put it as “MySQL Killer” especially until it is out of Beta and we get more hands-on experiences with it to understand Its limits.

Infobright, NitroDB, Kickfire all promise great speed improvements for certain kind of operations and I’m quite confident things work this way for queries they have on presentation slides and in demos, however you always see best results on such marketing presentation.

I think Kickfire has a great idea of hardware accelerator – I just do not have enough information at this point to judge how well they have implemented it in practice.

Kickfire is very fast for what it does, but you should consider couple things here.

First – KickFire is positioned as Datawarehouse engine right now – it is not designed for OLTP etc.

Second – MySQL just handled these particular queries very badly. In real MySQL world if you have such queries you either rewrite your queries or you do not use MySQL.