Comments on: Air traffic queries in LucidDB http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/ Percona's MySQL & InnoDB performance and scalability blog Sat, 11 Feb 2012 16:45:54 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Vadim http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-698207 Vadim Fri, 18 Dec 2009 17:49:57 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-698207 Otis, John, That's OK. I can't spend more time on SSB + ontime queries benchmarks, but I will be able provide hardware access for next benchmark. Otis, John,

That’s OK. I can’t spend more time on SSB + ontime queries benchmarks, but I will be able
provide hardware access for next benchmark.

]]> By: Otis Gospodnetic http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-698185 Otis Gospodnetic Fri, 18 Dec 2009 17:14:43 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-698185 What John wrote above makes sense: +1 Without that, the experts will always be a little handicapped and the benchmark quality will suffer. Thank you for doing this valuable work! What John wrote above makes sense: +1
Without that, the experts will always be a little handicapped and the benchmark quality will suffer.

Thank you for doing this valuable work!

]]> By: John Sichi http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-698158 John Sichi Fri, 18 Dec 2009 16:40:18 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-698158 We're very happy to have a well-respected independent organization like Percona running these. As one of the "experts" involved, one thing I can say is that it would be great if we could be given direct access to the box during the setup phase to help with tuning, and then release it to Vadim for exclusive access/auditing of the final runs. Doing everything by proxy adds enough overhead that it's not always possible to try as many configurations as one would like. Working with big data sets increases the challenge due to the times involved (particularly for loading), so starting with a small validation set first can help a lot too before moving on to full scale. I realize there are other issues like security and multi-user contention on shared hardware that may not make direct access practical. We’re very happy to have a well-respected independent organization like Percona running these. As one of the “experts” involved, one thing I can say is that it would be great if we could be given direct access to the box during the setup phase to help with tuning, and then release it to Vadim for exclusive access/auditing of the final runs. Doing everything by proxy adds enough overhead that it’s not always possible to try as many configurations as one would like. Working with big data sets increases the challenge due to the times involved (particularly for loading), so starting with a small validation set first can help a lot too before moving on to full scale. I realize there are other issues like security and multi-user contention on shared hardware that may not make direct access practical.

]]> By: Otis Gospodnetic http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-698121 Otis Gospodnetic Fri, 18 Dec 2009 15:22:31 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-698121 Excellent! This is good, because benchmarks done in a vacuum are often not realistic, unless the person preparing the benchmark is truly an expert in all tools being benchmarked, which is very rarely the case. I know I've seen such expert-advice/review-free benchmarks in search engine land (Lucene, Solr, Sphinx, Xapian...) and, as a Lucene/Solr guy, found them frustrating and worthless. Is all the code/setup used for benchmarking going to be available, so individuals and other organizations can make use of it, possibly improving it, and even maintaining it, so that it is easy to re-run the benchmark with newer releases of the software? Putting everything on Github, where it can be easily forked, may make sense here. Excellent!
This is good, because benchmarks done in a vacuum are often not realistic, unless the person preparing the benchmark is truly an expert in all tools being benchmarked, which is very rarely the case. I know I’ve seen such expert-advice/review-free benchmarks in search engine land (Lucene, Solr, Sphinx, Xapian…) and, as a Lucene/Solr guy, found them frustrating and worthless.

Is all the code/setup used for benchmarking going to be available, so individuals and other organizations can make use of it, possibly improving it, and even maintaining it, so that it is easy to re-run the benchmark with newer releases of the software?
Putting everything on Github, where it can be easily forked, may make sense here.

]]> By: Vadim http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-697906 Vadim Fri, 18 Dec 2009 06:16:51 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-697906 Otis, I have review of the results from experts from each vendor. There are some comments from them, but nothing significant that could change final results noticeably. Otis,

I have review of the results from experts from each vendor.
There are some comments from them, but nothing significant that could change final results noticeably.

]]> By: Otis Gospodnetic http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-697904 Otis Gospodnetic Fri, 18 Dec 2009 06:13:21 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-697904 Thanks for the pointer, Vadim, I can't wait to see the final results! When doing these comparisons, are you getting experts from each project to tune/optimize their DB? Thanks for the pointer, Vadim, I can’t wait to see the final results!
When doing these comparisons, are you getting experts from each project to tune/optimize their DB?

]]> By: Vadim http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-697903 Vadim Fri, 18 Dec 2009 06:10:18 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-697903 Otis, I just run SSB benchmark, but results are not officially published yet. You can see results here http://www.percona.com/docs/wiki/benchmark:ssb:start Also more benchmarks coming. Otis,

I just run SSB benchmark, but results are not officially published yet.
You can see results here
http://www.percona.com/docs/wiki/benchmark:ssb:start

Also more benchmarks coming.

]]> By: Otis Gospodnetic http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-697901 Otis Gospodnetic Fri, 18 Dec 2009 06:04:12 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-697901 You are right, 0.9.2 is the latest. Are you planning on doing another test with LucidDB? I'd be interested in seeing the comparison of LucidDB, MonetDB, and InfiniDB (they just had a release). Thanks. You are right, 0.9.2 is the latest.
Are you planning on doing another test with LucidDB? I’d be interested in seeing the comparison of LucidDB, MonetDB, and InfiniDB (they just had a release).

Thanks.

]]> By: Vadim http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-697899 Vadim Fri, 18 Dec 2009 05:58:53 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-697899 Otis, It was 0.9.1. I see latest available release 0.9.2 Otis,

It was 0.9.1.
I see latest available release 0.9.2

]]> By: Otis Gospodnetic http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/comment-page-1/#comment-697499 Otis Gospodnetic Thu, 17 Dec 2009 12:33:29 +0000 http://www.mysqlperformanceblog.com/?p=1537#comment-697499 Hm, LucidDB doesn't look good in this benchmark. Was the test done with LucidDB 0.9.4 or some earlier version? Hm, LucidDB doesn’t look good in this benchmark. Was the test done with LucidDB 0.9.4 or some earlier version?

]]>