… not used to process the data, at least not on the large scale – instead it is used for data storage and light duty data retrieval. Even… scale data processing other techniques are used. Google uses some bigtable based technologies for search and Google Analytics (which is quite typical log processing…
Post: State of MySQL Market and will Replication live ?
April 9, 2008 By 29 Comments
… and is abut not SQL and non relational data processing and storage system – BigTable with MapReduce, Amazon Dynamo and SimpleDB, Hadoop Another angle of customization of data store and processing was using non SQL and…
Post: Data mart or data warehouse?
July 15, 2010 By 13 Comments
… aggregation process. Comparing the performance of OLAP with and without aggregation over multiple MySQL storage engines at various data scales. What is a data warehouse…=”Typical star, snowflake and data warehouse schemata.”][/caption] Is one better than the other? In the game of data warehousing, a combination of…
Post: Why MySQL could be slow with large tables ?
June 9, 2006 By 139 Comments
… respect and what works well for Oracle,MS SQL, PostgreSQL may not work well for MySQL and other way around. Even storage engines… with large data sets and complex queries here are few tips Try to fit data set you’re working with in memory – Processing in memory is so much faster and you have whole…
Post: Why you should ignore MySQL's key cache hit ratio
February 28, 2010 By 27 Comments
… on a ratio of counters, because the process of dividing one counter by the other … both servers are identically tuned, and they have identical data, hardware, and workload. Even that doesn’t help,…mixture of storage engines (some InnoDB, some MyISAM, which is more important to you, etc) and choose …
Post: Using LVM for MySQL Backup and Replication Setup
August 21, 2006 By 53 Comments
… local disk based storage engines It works with MyISAM and Innodb and BDB, It also should work with Solid, PrimeXT and Falcon storage engines. Fast… for Logical Volume Main/Data . You should specify enough of undo space to hold modifications during backup process – I’ve specified 16GB…
Post: Recovery beyond data restore
August 2, 2008 By 11 Comments
… data restore process you better look at the whole process which is required to bring system to the working state, including data consistency requirements and… mode for large data sets. Such global recovery can also put very high stress on your network and backup storage and take a lot…
Post: Shard-Query turbo charges Infobright community edition (ICE)
May 6, 2011 By 22 Comments
… it works with existing MySQL data sets and queries. Another advantage is that it works with all MySQL storage engines. This set of… parallelism is possible and very large data sets may be processed. As each machine examines only a small subset of the data, performance can… 2GB total data. then spread over six servers). Due to ICE limitations, the data was split before loading. The splitting/loading process will…
Post: The Doom of Multiple Storage Engines
May 8, 2010 By 22 Comments
… and development complexity. Let me touch just on the few complications it brings in Conversions – Each storage engine has its own data storage format so MySQL has to do a lot of copying and conversions while accessing the data, which… could save a lot of CPU cycles by having storage format same as processing format. We could tune Optimizer to handle Innodb…
Post: High-Performance Click Analysis with MySQL
December 22, 2008 By 13 Comments
…I’ve been involved with, having day-old data is adequate, and users don’t expect realtime. The …pretty much inevitable: if you do this kind of data processing in MySQL, you’re going to end up heavily… An impression per day becomes a fixed overhead of storage size. So, you actually have as many …
