…, but QA still needs to have some tables in fresh status. – resharding databases, moving some shards from one server to another. So… still not so easy as with MyISAM tables when you just copy table.frm, table.MYD, table.MYI files, but so nice here – just can copy InnoDB tables in fully non-blocking mode…
Post: Using any general purpose computer as a special purpose SIMD computer
May 16, 2011 By 5 Comments
… expensive process unless the work can be done in parallel. Shard-Query introduces set based processing, which on the surface appears… set multiple times. mysql> load data infile ‘/tmp/words’ into table words (chars); Query OK, 6033 rows affected (0.01 sec… threaded operation burns one CPU. To help completely demonstrate how Shard-Query makes parallel set operations work, I’ll operate in…
Post: High-Performance Click Analysis with MySQL
December 22, 2008 By 13 Comments
… things that seem “less advanced” than using a database. Sharding and Partitioning Sharding is inevitable if your write workload exceeds the capacity… replication, the capacity of a single slave). Sharding can also help you avoid massive tables that are too big to maintain. If… databases. In some cases I’ve been able to avoid sharding for a long time and run on many fewer disk…
Comment: Why you don't want to shard.
November 22, 2009 By 34 Comments
… having to resort to sharding. Also, Sharding only works if the shards are largely independent, eg. GMail user accounts. But sharding and an integrated… slow it down as the shards need to communicate. Some databases (Oracle) can horizontally partition a table (and I hope thus a… right approach. Keep the logical/physcial separation. Google style sharding and big table are a hack that ignores 50 years of database…
Comment: Why you don't want to shard.
August 6, 2009 By 34 Comments
I agree with the overall statement that sharding is often overkill, I’m not as fond of partitioning … scans)? Alternatively, with sharding, one can still join various tables other if the sharding was done by a good key. Sharding keeps much of… make sense in many or even most cases, but that sharding can make more sense in a relationship rich application where…
Comment: Scaling to 256-way the Sun way
November 12, 2008 By 13 Comments
…, alter table etc) people have to shard sooner than they would have and if MySQL would be more scalable. Second – when sharding number… running on the different servers because of shared hardware. People sharding on middle end will much rather have less MySQL instances… with than 50. On the higher end – dealing with 100 shards or with 1000 does not really matter because you have…
Comment: Finding your MySQL High-Availability solution – The questions
October 19, 2009 By 36 Comments
…) adding new ndbd nodes and of course doing alter tables. For alter tables, only replication can reduce the downtime. I agree that… cannot lose data, ruling out replication, and needs HA and sharding. The only solution there is to have multiple Heartbeat/DRBD… can be built like yours and for existing ones, implementing sharding in the application might a complex and costy task. For…
Post: State of MySQL Market and will Replication live ?
April 9, 2008 By 29 Comments
… features out of the box started deploying their custom solutions – Sharding and Caching. Indeed Memcache is the most known caching technologies… invalidation and consistence issues is not fun. Neither they like sharding with requirement to split the data in defined way so… processing was using non SQL and not systems – Google Big Table, Amazon Dynamo and SimpleDB, Hadoop, CouchDB which comes computed with…
Post: Upgrading MySQL
January 5, 2010 By 24 Comments
… and made it to caught up you can use mk-table-checksum to ensure data is the same. It is possible… in case you’re having Sharded envinronment you often approach things a bit differently – upgrading one shard with full validation and when doing reckless upgrades for other shards after you’re sure your…
Comment: Why you don't want to shard.
August 14, 2009 By 34 Comments
… example hosting million of separate blogs is very easy to shard because there is no interdependencies. The large data also indeed… due to challenges with backups and expecially things as alter table. http://www.mysqlperformanceblog.com/2006/10/08/small-things-are… will be consistent (even though not up to date) – in sharded envinronment backups will correspond to different point in time and…
