…. Even single-row writes are likely to be slower in PARTITIONed tables unless you include something to facilitate pruning. A billion rows in a BTree is only (about) 5 levels deep. If you split that table into 100 partitions, the BTree in each partition would be about 4 levels. A “point query” in a billion-row (non-partitioned) table…
Comment: Percona Server 5.5.30 with TokuDB for MySQL
April 28, 2013 By 26 Comments
… table to TokuDB: “ERROR 1030 (HY000): Got error -1 from storage engine” The table has approximately 1.6 billion rows and I used “ALTER TABLE… to that, I was successful in converting a 10 GB table with a similar schema.
Post: Benchmarking single-row insert performance on Amazon EC2
May 16, 2012 By 17 Comments
… iiBench which has been developed by Tokutek. Though the “1 billion row insert challenge” for which this tool was originally built is… writing to the table, with each instance of iiBench writing 200 million single row inserts, for a total of 1 billion rows. I ran the test both with the table purchases_noindex which…
Post: Why MySQL could be slow with large tables ?
June 9, 2006 By 141 Comments
… installations with many billions of rows and delivers great performance. What could be the reason ? The reason is normally table design and understanding… rows (12GB) table was scanned in less than 5 minutes. Now if we would do eq join of the table to other 30mil rows table and it will be completely random. We’ll need to perform 30 millions of random row reads…
Comment: Predicting how long data load would take
March 26, 2012 By 14 Comments
One trick for really huge, hash-partitioned InnoDB tables is to sort by the partition rule, then by the … very nice buffer pool behavior. We loaded a 1.5 billion row table in “only” 30 hours using this approach.
Comment: Benchmarking single-row insert performance on Amazon EC2
May 16, 2012 By 17 Comments
… entire table fits in main memory. Each row is on the order of 30 bytes in the primary table, and so a billion rows is…. With a 55GB buffer pool, we would expect the primary table to simply fit. But with all 4 indexes the data…. Tim’s measurements (http://www.tokutek.com/2012/01/1-billion-insertions-%E2%80%93-the-wait-is-over/) on a…
Comment: MySQL Partitioning - can save you or kill you
June 28, 2012 By 45 Comments
… because of your “billion” rows. > INT(4) — the 4 says nothing. The INT will be 4 bytes, range of +/-2 billion. Possibly overkill… MERGE). Your first two SELECTs may benefit from designing “summary table(s)”, populating them as the data arrives, then SELECTing against…, which has less overhead than utf8. Let me guess… SHOW TABLE STATUS says 50GB of data + 30GB of index (MyISAM) or…
Post: New OLAP Wikistat benchmark: Introduction and call for feedbacks
January 28, 2010 By 16 Comments
…) we have: Table pages: 724.550.811 rows. data size: 40476M Table datesinfo: 9624 rows, one entry represents 1 hour Table projects: 2025 rows Table pagestats Data for 2009-06: # 3.453.013.109 rows / size….618 rows / size 68152M So with two months of stats we have about 172GB of data with about 7 billion rows in fact table…
Comment: Star Schema Bechmark: InfoBright, InfiniDB and LucidDB
February 19, 2010 By 48 Comments
… SF-10,000 and a SF-1,000 (60 billion and 6 billion facts respectively) into one InfiniDB instance on a 6… fact table. Anyway, further details are available at: http://www.infinidb.org/myblog-admin/mysql-parallel-query-processing-of-60-billion-rows-via…
Comment: Why MySQL could be slow with large tables ?
November 21, 2009 By 141 Comments
Hello all, I have a very large table, 3 billion rows, 300GB in size. It’s of ip traffic logs. Select … will most likely have to break this very large table down into tables by week. My question is what my plan of… to get the best insert performance? MERGE tables? PARTITION tables? Simply break up my big table into smaller ones? I think the root…
