We have a lot of customers who do click analysis, site analytics, search engine marketing, online advertising, user behavior analysis, and many similar types of work. The first thing these have in common is that they’re generally some kind of loggable event. The next characteristic of a lot of these systems (real or planned) is [...]
Beware of MyISAM Key Cache mutex contention
August 12, 2008 By 10 Comments
Today I was working with the client loading data to MyISAM tables at very high rate. Hundreds of millions rows are loaded daily into single MySQL instance with bursts up to 100K of records/sec which need to be inserted (in the table with few indexes). It was good not all records had to go to [...]
How to load large files safely into InnoDB with LOAD DATA INFILE
July 3, 2008 By 29 Comments
Recently I had a customer ask me about loading two huge files into InnoDB with LOAD DATA INFILE. The goal was to load this data on many servers without putting it into the binary log. While this is generally a fast way to load data (especially if you disable unique key checks and foreign key [...]
Quickly preloading Innodb tables in the buffer pool
May 1, 2008 By 9 Comments
In the previous post I mentioned a way I use to preload Clustered Index (data) for Innodb tables. Though I thought this topic would benefit from a bit more information. But lest first start with feature request for Innodb Team: All ways I mention here are hacks and they can’t be as efficient as native [...]
Predicting how long data load would take
May 24, 2007 By 14 Comments
I had this question asked many times during last week, and there is really no easy answer. There are just way too many variables to consider especially while loading large schemas with a lot of different table structures. So which variables affect the load speed: Table Structure This one is tricky. The shorter rows you [...]
MySQL Server Variables – SQL layer or Storage Engine specific.
June 8, 2006 By 14 Comments
MySQL Server has tons of variables which may be adjusted to change behavior or for performance purposes. They are documented in the manual as well as on new page Jay has created. Still I see constant confusion out where which of variables apply to storage engines only and which are used on SQL layer and [...]
