There is significant portion of customers which are still using MyISAM when they come to us, so one of the big questions is when it is feasible to move to Innodb and when staying on MyISAM is preferred ?

I generally prefer to see Innodb as the main storage engine because it makes life much simpler in the end for most users – you do not get to deal with recovering tables on the crash or partially executed statements. Table locks is no more problem, hot backups are easy, though there are some important things which we have to consider on case by case basics before recommending the move.

Is MyISAM used as default or as a choice ? This is the most important question to ask upfront. Sometimes MyISAM is there just because it is default, in other cases this is deliberate choice with system being optimized to deal with MyISAM limits, for example there is a dedicated slave available for all long reporting queries. In case MyISAM was chosen not just happened to be it is important to build the good argument to suggest Innodb.

Application Readiness Application should be ready to work with Innodb, for example be ready to deal with deadlocks which can happen with Innodb even if you do not use transactions, but which are not existent with MyISAM. QA has to be performed as part of the move.

Performance Innodb has a lot to offer in terms of performance – Performance benefits and drawbacks. On the benefits side we usually see clustering by primary key, caching data, higher concurrency, background flushes while on the drawbacks side we see significantly large table size (especially if data size is close to memory size), generally slower writes, slower blob handling, concurrency issues, problems dealing with very large number of tables, slow data load and ALTER TABLE and others. Another big one is COUNT(*) without where clause which is often the show stopper for them move until it is worked around.

Operations What is good for MyISAM kills Innodb, such as copying binary tables between the servers. It is important the team understands Innodb and knows how to handle it, or be able to learn it. It is also important to adjust processes as required to work with Innodb. For example binary copy of one of the databases from the Slave to the dev envinronment works great for MyISAM but does not work with Innodb. Backup tools like “mysqlhotcopy” does not work etc. Note Performance also affects Operations aspects a lot – for example using mysqldump as a backup may well work for MyISAM but will start taking way too much time to do restore for Innodb. On large scale installations mysqldump does not work anyway but it may still work for you when you’re running MyISAM but instantly break upon upgrading to Innodb.

Features The MyISAM features which forbid moving to Innodb are typically Full Text Search and RTREE indexes/GIS with Full Text being much more common. There are workarounds for both of them, including dedicated MyISAM slave or shadow table but it is important to consider them.

How about Mixing Storage Engines ? Sure you can mix storage engines but I suggest you doing is wisely. It complicates operations tasks (backups, balancing, performance analyzes) as well as it exercises not so common paths in the MySQL server – in particular Optimizer may have harder time because costs between storage engines may not be well balanced or replication of mixed table types which is quite complicated.

I prefer to pick one storage engine (typically Innodb) and when use other tables when it really gives substantial gains. I would not switch table to MyISAM because it gives 5% performance improvement but I can perfectly use MyISAM (or Archive) for logging.

Innodb Needs Tuning As a final note about MyISAM to Innodb migration I should mention about Innodb tuning. Innodb needs tuning. Really. MyISAM for many applications can work well with defaults. I’ve seen hundreds of GB databases ran with MyISAM with default settings and it worked reasonably. Innodb needs resources and it will not work well with defaults a lot. Tuning MyISAM from defaults rarely gives more than 2-3 times gain while it can be as much as 10-50 times for Innodb tables in particular for write intensive workloads. Check here for details.

Note: As Few people questioned me, I indeed forgot to clarify the scope here – I’m mainly speaking about OLTP/ Traditional web applications. for Analytics things are a lot different.

61 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Dale

We evaluated switching to InnoDB.

Performance vs. MySQL was horrible, even after tuning. SELECT performance is crucial and it just did not hold up.

Management of the massive InnoDB binary file was daunting unlike MyISAM which has each table in its own set of files.

The show stopper was restore of tables. If we needed to do a restore of any table in InnoDB, we would need to restore the full InnoDB database from tape, move it to another sql server and then find what we are looking for (at least that seems to be the flow of things to be done for such). In our environment, some of our tables are hundreds of gigs and our total database is much larger – the time to do this is just unacceptable.

To be real honest, I don’t know why a production shop would use InnoDB based on what we have experienced thus far. It is slow, hard to manage and not at all viable in a DR situation when you have to quickly restore a single database or table.

I would be interested in other comments in regard to the DR aspect, am I correct in the steps necessary to restore a single table from tape?

Ernesto Vargas

I have seen many cases were people use MyISAM because of the Full-Text and they don’t know other alternatives…Sphinx Search Engine for example.

Most people that use MyISAM may lack the knowledge and expertise to manage a full blow transaction database engine like InnoDB. At the end what matter is that the data is safe, trustable and reliable and you cannot guarantee that with MyISAM on site with medium concurrency.

Peter. What is your best method for InnoDB backup on hunders of GB datasets?

Kevin Burton

I think the biggest hurdle for switching is the massive 3-4x database bloat that comes with InnoDB.

You guys might want to take a stab at solving that problem. I haven’t looked at the internals much but my hunch is that the on disk performance efficiency could be improved.

DaveB1

Interesting topic, but difficult to read. Is your native language not English? I would suggest having someone running through your article to correct bad grammar and punctuation before posting.

Fineas K.

The right tool for the right job. I don’t think this is a wise recommendation. Both MyISAM and InnoDB have their place. I tend to use both where they accel and would never remove either as a viable option.

Ms. Anthrope

@Dale
You likely needed better tuning – hire a good MySQL DBA and/or consultant.

As for restores, selective restoring from binary files is painful in InnoDB. If you weren’t using LVM and doing raw backups w/ InnoDB, you probably weren’t backing your databases up properly either. For per-table restores, we use mysqldump + –single-transaction and splice out what we need with some internal tools when the need arises.

David

Peter,
I think you mostly talk about OLTP between MyISAM and InnoDB
However, you have to realize that a lot of people has use MySQL in their OLAP application or hybird operation.
In that case, I think MyISAM has the advantage.
I also look forward to the new Maria storage engine, which is transaction safe engine.

surf

typo?
with deadlocks which can happen with Innodb even if you do not use transactions, but which are not existent with Innodb.

Dale

@Anthrope you state:
> You likely needed better tuning – hire a good MySQL DBA and/or consultant.
We hired and discussed this with a certified MySQL consultant from Sun – he agreed with our cons and with how we had tuned our system.

> As for restores, selective restoring from binary files is painful in InnoDB. If you weren’t using LVM and doing raw backups w/ InnoDB,
> you probably weren’t backing your databases up properly either. For per-table restores, we use mysqldump + –single-transaction and
> splice out what we need with some internal tools when the need arises.

Right, try doing a mysqldump and splicing out what you need when maybe a critical table of data needs to be restored quickly and your ops is down at a rate of $5k per minute. The above would take us hours of time costing us hundreds of thousands of dollars potentially. The point being, I have to plan on a “really bad day” hoping it will never occur. It is easily in the realm of possibility that we would have to restore a table from tape to deal with some kind of unknown situation. InnoDB was not built for this scenario. The other person’s idea of using the InnoDB feature of one file per table was good, but appears to be very unnatural and hard to manage?

I realize many “top sites” use InnoDB. I would be curious to know why they decided such and how they handle the query throughput issue and of course their DR plans for a critical failure and need to restore a table rapidly.

Thanks for the article!

D

Shlomi Noach

@Dale,

Hi,

1. Most DBAs and companies (I’ve met and seen a few) do not need to do single table restore on a daily (weekly?) basis. Actually, there shouldn’t be a reason to do so in the first place. I would suggest that there is some design flaw, or otherwise programmatic bug, which leads you to this need.

2. You speak of volumes of 100GB and above, probably you have more than 1TB of data. You say you cannot afford the restoration time from a mysqldump’s single table parsing.
In that case, I suspect you will certainly NOT have the time for a REPAIR TABLE on your MyISAM’s corrupted table. You say you need to prepare for a “really bad day”. I’ve met a DBA who has many TB of data, all in MyISAM. On average, 5 days each week are consumed by REPAIR TABLE. The larger tables take 4-5 *days* to restore.
Please be prepared for a very bad day. It *will* come.

3. I’m not suggesting InnoDB is right for you, since I don;t know anything about your application. Perhaps MyISAM is the right choice. You’ve has a review by a MySQL consultant. I’m assuming the review was professional.
But you must remember that your application is probably tuned to work with MyISAM. Just as your application would not work well with Oracle right now, there’s always adjustments to the types of queries you run against the storage engine.

4. InnoDB is far from being perfect.

5. Are you using a Unix-like OS? If so, what file system are you using? Unjournalled, like ext2, or journaled, like ext3, xfs? If you’re running Windows, do you useFAT32 of NTFS?
I suspect you’re using journaled file systems, although they are *slower*. The reason is – because they’re safer. MyISAM’s crashes are a show stopper for myself.

6. “…To be real honest, I don’t know why a production shop would use InnoDB based on what we have experienced thus far. …”
I think Peter summed it up very well in comment #18.

If the above sounded like a “always use InnoDB” preach, that was not my intent. I was merely trying to point some issues with regard to your post.

As a final suggestion, please read Peter’s comments again, they are very helpful.

Best regards,
Shlomi

Baron Schwartz

About performance — there are many cases when InnoDB can outperform MyISAM significantly, especially when you know how to use its clustered index and how to take advantage of index-covered queries.

There is no way to say one is faster than the other. It always depends on the specific case.

Log Buffer

On the MySQL Performance Blog, Peter Zaitsev’s question is, should you move from MyISAM to Innodb? Peter and his readers lay out the criteria he think are at play when answering this question for yourself.

Log Buffer #131

mike

We are having performance difficulties with the innodb plugin from innodb.com , using ver. 1.01 and mysql 5.1.24:

Our 15 tables are partitioned (with 1024 partitions each), the innodb_file_per_table is on, and data is compressed (key-buffer-size=4). Total db content is only 15G.
Our Inserts drop to 2500 samples / seconds (400 kB per second) when compression is on. We perform bulk inserts (i.e. values (),(),(),…).
I set the innodb_buffer_pool_size to 1G (having 4G ram) which seems to be the highest buffer allowed. if exeeded, mysql does not recognise the search engine (!) anymore.
Removing the compression helps for insert performance.

Create table and alter table statements are very slow too if we have partitions (indep. from compression).
Without partitions, the innodb plugin was o.k., slower than MYISAM (incl. inserts, selects) but still o.k.

Unfortunately we need the compression because the db (goal: several TB) may be populated with a lot of redundant data.

Any idea? would falcon be an alternative?

Shlomi Noach

Mike,

This one has a 0.999 probability of being the solution to your 1GB problem:
Install a 64bit linux.

I suspect you have a 32bit installation, which is why the storage engine would not accept more than 1GB: it’s out of address space. In total, you may squeeze some 1.5G from all MySQL components together…

On a 4GB machine, setting innodb_buffer_pool_size to 2.5G (assuming only InnoDB is used, machine is solely MySQL) is probably a reasonable value, which leaves you much enough space for complicated queries and many connections.

I suspect that moving from 1GB to 2.5GB will increase you InnoDB performance a lot, based on the fact you only have 15GB of data.

Regards

mike

Hi there,
We are using this many partitions as the application most probably will run for around 10 years and gather data for very long timespans. In order to facilitate pruning, we chose the maximum amount of partitions per table, which currently corresponds to 3,5 days per partition (partition by range). Best case scenario would have been if would have been able to compress this data aditionally.
Yes, our OS is SLES 9 32bit. Maybe we could change something there…
Today we experienced a good performance increase if we changed from Barracuda back to Antelope file format. We do not have compression then, but InnoDB plugin performs with around 13000 samples/second on inserts compared to 2000-3000 with Barracuda and compression.

cheers

Roman Lvov

@Peter
>Yes I speak mostly about OLTP. OLAP with full table scans and similar stuff indeed is the good usage for MyISAM. I should have been more clear with that.

Uff, ok, good. I was surprised to hear that InnoDB is almost always better choice than MyISAM. Because I’m that guy who develops OLAP system. My DBs are very small compared to the mentioned above cases – mysqldump of the largest DB is only 550M. But queries upon data can run for seconds and even tens seconds (I even had one which ran 3 mins before optimization), as each query joins several tables, often 5-6-7 ones. And when the number of rows in the two biggest tables (which participate in almost every query) reached 300K and 50K, performance issues are visible. Data updates are performed once a day, and can be re-done if in the case of a crash entire DB has to be restored from a daily backup, so I almost perfectly live without InnoDB. So I think I should stick to MyISAM.
I can guess that now, when you started developing XtraDB based on InnoDB, you will be almost entirely focused on InnoDB issues. It would be really great if you find some time to continue covering MyISAM as well.

Vladimir Rusinov

Hi!

I’ve translated it to russian: http://greenmice.info/ru/node/106

HTH to any of my russian colleagues.

Ed

The deal breaker: InnoDB does not support AUTO_INCREMENT in positions other than the 1st.

If you have an app that uses that MyISAM feature porting it to InnoDB can be a major headache.

Example:

#############################################
# MyISAM

CREATE TABLE MyISAM_animals (grp ENUM(‘fish’,’mammal’,’bird’) NOT NULL, id MEDIUMINT NOT NULL AUTO_INCREMENT, name CHAR(30) NOT NULL, PRIMARY KEY (grp,id))ENGINE=MyISAM;

INSERT INTO MyISAM_animals (grp,name) VALUES (‘mammal’,’dog’),(‘mammal’,’cat’),(‘bird’,’penguin’),(‘fish’,’lax’),(‘mammal’,’whale’),(‘bird’,’ostrich’);

SELECT * FROM MyISAM_animals ORDER BY grp,id;
+——–+—-+———+
| grp | id | name |
+——–+—-+———+
| fish | 1 | lax |
| mammal | 1 | dog |
| mammal | 2 | cat |
| mammal | 3 | whale |
| bird | 1 | penguin |
| bird | 2 | ostrich |
+——–+—-+———+

#############################################
# InnoDB

CREATE TABLE InnoDB_animals (grp ENUM(‘fish’,’mammal’,’bird’) NOT NULL, id MEDIUMINT NOT NULL AUTO_INCREMENT, name CHAR(30) NOT NULL, PRIMARY KEY (grp,id))ENGINE=InnoDB;
### ERROR 1075 (42000): Incorrect table definition; there can be only one auto column and it must be defined as a key

CREATE TABLE InnoDB_animals (grp ENUM(‘fish’,’mammal’,’bird’) NOT NULL, id MEDIUMINT NOT NULL AUTO_INCREMENT, name CHAR(30) NOT NULL, PRIMARY KEY (id,grp))ENGINE=InnoDB;

INSERT INTO InnoDB_animals (grp,name) VALUES (‘mammal’,’dog’),(‘mammal’,’cat’),(‘bird’,’penguin’),(‘fish’,’lax’),(‘mammal’,’whale’),(‘bird’,’ostrich’);

SELECT * FROM InnoDB_animals ORDER BY grp,id;

+——–+—-+———+
| grp | id | name |
+——–+—-+———+
| fish | 4 | lax |
| mammal | 1 | dog |
| mammal | 2 | cat |
| mammal | 5 | whale |
| bird | 3 | penguin |
| bird | 6 | ostrich |
+——–+—-+———+

#############################################

Shlomi Noach

@Ed (31)

Why would this be a deal breaker?

Tom

To call MyISAM a database is a joke. To have something not crash safe and not supporting ACID transactions and call it “database”, you must have missed something in your education. Better compare it to cache products, and then the speed is not very impressive either. DRAM prices have changed the game in ways the DB industry have yet to fathom. DB devs are years behind in optimizing the sweet DRAM/DISK relation point.

haram

is there a way to do full text search using InnoDB

MikeTrest

Business Continuity / Disaster Recovery using InnoDB? We use an internal process based on mysqldump and innodb file-per table.
Our “Database File-Per-Table Archives” web site summarizes like this: “31 Hosts 90 Databases 2919 Tables 91 Dates 354456 Backup Files”

The 31 Hosts are the Master hosts only. Add another 67 slaves in groups replicating upto 6 slaves machines per group. Largest instances top out at approx 60M rows. Most tables are < 0.5M rows. Some applications use MyISAM with partitioned tables. Since every machine is a dedicated instance, mixing engines is not an issue. All DBs are fronted by memcached machines (64 instances). All applications are written to use memcache before hitting DB. We use RAID0 with multiple disks on all SLAVES. MASTER machines use RAID5. Ratio of reads-to-writes is 7,000-to-1. Collectively, about 600M DB transactions per day after peeling off 78% of the reads on the memcaches.

Any table can be selected from the backup archive with a couple of clicks to choose date,table, and host instance. It can be restored to a master with a few more clicks by appropriately authorized DBA. For failures on a MASTER, we do a quick shift and make one of the slaves the new master. We then repair/reload the new master and SYNC it from one of the slaves before switching it back to MASTER. Slaves are simply rebuilt and added back to the mix when their SYNC is completed. Therefore, the HUGE/SLOW restore of large InnoDB tables is not an issue,

My suggestion to use all the tools and techniques available to you, icrease the use of MASTER-SLAVE replication, and use file-per-table management. Then restore-from-backup can be done a comfortable pace.

Bacon Legs

I really wish I could understand more than 60% of this article. PLEASE have a native English-speaker translate what the hell you are trying to say. It’s almost as annoying to read as it is to see you tell people in the comments to “skip the intelligible parts” – if you’re too important to take the time to write proper English, how about writing your articles in another language?

incognito

@Bacon Legs
If you are so sensitive to English, maybe you should skip more than half of the www, not only this article. If you would have learned some other languages (other) than English, you wouldn’t be so academic and sniffy and you would have understood what Peter wrote.

DotCom

I laughed the other day, reading on daniweb how “myisam” is faster than “innodb”. Maybe the need some kind of literature to clear their mind. myisam is for people who don’t know what they are doing, saying: “ok, i’ll just go with default”

http://www.mysqlperformanceblog.com/2007/01/08/innodb-vs-myisam-vs-falcon-benchmarks-part-1/

InnoDB is even faster nowadays.

Eric Gillette

Well, in my experience, performance-wise on a website that is run by a database, or where parsing occurs using SSI which is pulling information from an InnoDB as opposed to MyISAM is a major difference. I’ve noticed much faster response with InnoDB as opposed to MyISAM — plus InnoDB doesn’t develop overhead like MyISAM does. But. . .that’s just my opinion!

I can say for sure that as a PHP developer, I will be releasing ALL of my apps from now on with InnoDB assumed, and MyISAM as backup support if InnoDB doesn’t exist on the server. . .

Dale Lancaster

Eric, what do you mean InnoDB doesn’t develop overhead like MyISAM? We run it all the time and see no overhead/slowdown and we are talking millions of rows. Just curious the full thought behind the statement.

thanks

dale

RamAM

Solid article Peter.
FYI I’m not a native English speaker and I had zero problem understanding your article. Also if I had trouble understanding it I would consider that my problem not the universe’s and not yours 😉

cheers

Sam

I am a native speaker and I understood it, but it was painful to read. Good work on publishing the info, and don’t mind the grammar police, they are just French descendants.

jayaram

I am working with one table it contains 30million entries about 3GB, which is MyISAM engine.
And my application uses this table as so many of read and writes with in 1sec.

Problem is most of queries in locking stage, i.e problem with table level locking.

Can i move to InnoDB ? , What are the -ve and +ve things..

Could you please help me………………

Ernesto Vargas

@jayaram…

Do a SHOW VARIABLES and make sure have_innodb say YES. Then do an ALTER TABLE ENGINE INNODB and you will be it.

How many queries are getting stuck on with table level locking at peak time? With Innodb you will no have that problems since its ACID complaint.

Alex Yaroshevich

Неплохая статья) Спасибо.

Loc Dinh

Very nice article Peter, thanks for the information. Just one thing wasn’t mentioned and would be great if you can clarify for me:

My database has been running on MyISAM for a while and it does have multilingual data (Chinese, Japanese, French, English …), do you know any effect it might have to existing data if i changed to InnoDB?

Thanks

jayaram

@ Ernesto Vargas…

Thank you very much for your quick response :)
Yes , have_innodb is yes
At peak time minimum 50 to 60 read queries and 40 to 60 write queries on the same table.

Please give me my.cnf setting for innodb engine for this large data tables.
we are using 30 tables like this.

Please give me suggestions about working with innodb for these big tables.
And mostly me fear about backups…
Suggest me about better backup plans with innodb

Thank you

Ernesto Vargas

@jayaram…

Do a SHOW VARIABLES and make sure have_innodb say YES. Then do an ALTER TABLE ENGINE INNODB and you will be it.

How many queries are getting stuck on with table level locking at peak time? With Innodb you will no have that problems since its ACID complaint.

jayaram

I am working with one table it contains 30million entries about 3GB, which is MyISAM engine.
And my application uses this table as so many of read and writes with in 1sec.

Problem is most of queries in locking stage, i.e problem with table level locking.

Can i move to InnoDB ? , What are the -ve and +ve things..

Could you please help me………………

ponnusamyc

I am a native speaker and I understood it, but it was painful to read. Good work on publishing the info, and don’t mind the grammar police, they are just French descendants.