… BigData with a talk titled, “Conquering BigData with MySQL and Shard Query” Daniel Nichter, lead of the Percona Toolkit development team, will…
Post: Shard-Query turbo charges Infobright community edition (ICE)
May 6, 2011 By 23 Comments
… it allows Shard-Query to vastly improve the performance of queries which have to examine a large amount of data. Shard-Query includes a loader… multiple machines, scaling out to improve query response times significantly. These queries are a great test of Shard-Query features. It is currently approaching…
Post: Shard-Query EC2 images available
May 11, 2011 By 2 Comments
… run_query script will run one more more semicolon terminated SQL statements. The queries for the benchmark are in ~ec2-user/shard-query/queries.sql… will summarize the output from the ./run_query command, called pivot_results cd shard-query/ $ ./run_query < queries.sql | tee raw |./pivot_results & [1] 12359…
Post: Shard-Query adds parallelism to queries
November 15, 2010 By 18 Comments
… better) when running small numbers of queries which examine a lot of data. What is Shard-Query? Shard-Query was initially conceived as a utility to add parallelism to horizontally partitioned data sets by running queries against each…
Post: Distributed Set Processing with Shard-Query
May 14, 2011 By 27 Comments
… to give a quick preview, then explain exactly how Shard-Query works.Yes, Shard-Query scales very well at 20 nodes Distributed set processing… fastest incremental materialized view optimizations can be applied. Shard-Query works only on sets Shard-Query takes relational algebra to its logical maximum conclusion…
Post: Webinar: Building a highly scaleable distributed row, document or column store with MySQL and Shard-Query
February 6, 2013 By Leave a Comment
… Shard-Query” The first part of this webinar will focus on why distributed databases are needed, and on the techniques employed by Shard-Query… (massively parallel processing) database applications which can be deployed with Shard-Query and the performance aspects of each. The following types of…
Comment: Shard-Query turbo charges Infobright community edition (ICE)
October 5, 2012 By 23 Comments
…/libo/shard-query-read-only/include/shard-query.php on line 145 Notice: Undefined property: ShardQuery::$broadcast_query in /home/libo/shard-query-read-only/include/shard-query.php… in (1,2)”; $stmt = $shard_query->query($sql); if(!$stmt || $shard_query->error) { print_r($shard_query->error); } if($stmt) { while($row = $shard_query->my_fetch_assoc($stmt)) { print…
Post: Using any general purpose computer as a special purpose SIMD computer
May 16, 2011 By 5 Comments
… expensive process unless the work can be done in parallel. Shard-Query introduces set based processing, which on the surface appears to… threaded operation burns one CPU. To help completely demonstrate how Shard-Query makes parallel set operations work, I’ll operate in only… performance is a little worse. This is not unexpected. Since Shard-Query must add at small amount of overhead, a single threaded…
Post: "Shard early, shard often"
November 16, 2009 By 6 Comments
… gained by focusing on query/index/schema optimization. The gains from sharding are usually much lower. If you shard first, and then decide you want to tune query/index/schema to… would never recommend sharding to a customer until I had at least reviewed their slow query log with mk-query-digest and understood…
Post: Why you don't want to shard.
August 6, 2009 By 34 Comments
… have established that you need to shard there are quite a few options available to you: Sharding Partitioning by Application Function – This….) Sharding via a Lookup Service – This method works by having some sort of directory service which you query first to ask “what shard… out some of the more complicated sharding architectures. For example; another solution is to have shards all store fragments of data…
