Comments on: INSERT INTO … SELECT Performance with Innodb tables. http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/ Percona's MySQL & InnoDB performance and scalability blog Sat, 11 Feb 2012 16:45:54 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: mathew http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-867760 mathew Sun, 01 Jan 2012 18:45:49 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-867760 @josh: No this does not applies in MySQL 5.5+. I've already check that on a table containig 1 mln rows using insert into select from this table. No lock has been aquired and on the other thread i could simply update a row that should be blocked. But if you would like to have this exact behaviour you can use mentioned above: "lock in share mode" because this does the lock to be aqquired on the second table @josh: No this does not applies in MySQL 5.5+. I’ve already check that on a table containig 1 mln rows using insert into select from this table. No lock has been aquired and on the other thread i could simply update a row that should be blocked. But if you would like to have this exact behaviour you can use mentioned above: “lock in share mode” because this does the lock to be aqquired on the second table

]]> By: josh http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-865383 josh Wed, 28 Dec 2011 13:15:51 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-865383 +1 for @adelainev. I have the same question whether this behavior still applies in MySQL 5.5+. This post was written in 2006, so just want to see where things stand currently. +1 for @adelainev. I have the same question whether this behavior still applies in MySQL 5.5+. This post was written in 2006, so just want to see where things stand currently.

]]> By: adelainev http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-824661 adelainev Fri, 23 Sep 2011 12:00:57 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-824661 Does this behavior still apply to MySQL 5.5+ ? Does this behavior still apply to MySQL 5.5+ ?

]]> By: vishnu rao http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-823062 vishnu rao Sat, 17 Sep 2011 19:18:18 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-823062 hi peter, facebook urges people to be cautious while using "load data infile" in production. they report that many a time mysql had crashed. any thoughts on this ? many bugs have been also reported around this. http://www.facebook.com/note.php?note_id=394073200932 hi peter,

facebook urges people to be cautious while using “load data infile” in production. they report that many a time mysql had crashed.

any thoughts on this ? many bugs have been also reported around this.

http://www.facebook.com/note.php?note_id=394073200932

]]> By: Andrew Ferk http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-804630 Andrew Ferk Thu, 14 Apr 2011 19:46:36 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-804630 @Sylvain261 I tested SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SET binlog_format = MIXED; INSERT INTO ….SELECT … FROM …. and it works without error and does binlog as row. The issue with that is the potential for huge overhead. The INSERT INTO ... SELECT I'm running generates about 2.5 million records (or almost a GB in the binlog). Thank you. @Sylvain261

I tested
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET binlog_format = MIXED;
INSERT INTO ….SELECT … FROM ….
and it works without error and does binlog as row. The issue with that is the potential for huge overhead. The INSERT INTO … SELECT I’m running generates about 2.5 million records (or almost a GB in the binlog).

Thank you.

]]> By: jigen http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-797919 jigen Sun, 13 Feb 2011 06:05:04 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-797919 Hi, I am having problem with Selects ...between... timestamp.. for example: I have this select to select from a 200k rows... SELECT actionid FROM tablelogs WHERE timestampfield BETWEEN UNIX_TIMESTAMP('2010-10-01 ') AND UNIX_TIMESTAMP('2011-01-01') GROUP BY actionid.... this select is taking ages to process.... now, is there a way to optimize this select or speed up? The tablelog is innodb with index at actionid (INT). Also, if it will help, when I use the command select * from tablelogs vs. select actionid from tablelogs, the latter is taking ages as well while the one with * takes only 0.8 seconds. The table has 7 fields.... I am using MySQL 5+ in SQLYOG ... any ideas? thanks Hi,
I am having problem with Selects …between… timestamp..
for example:
I have this select to select from a 200k rows…
SELECT actionid FROM tablelogs WHERE timestampfield BETWEEN UNIX_TIMESTAMP(’2010-10-01 ‘) AND UNIX_TIMESTAMP(’2011-01-01′) GROUP BY actionid….

this select is taking ages to process…. now, is there a way to optimize this select or speed up?
The tablelog is innodb with index at actionid (INT).

Also, if it will help, when I use the command select * from tablelogs vs. select actionid from tablelogs, the latter is taking ages as well while the one with * takes only 0.8 seconds. The table has 7 fields….

I am using MySQL 5+ in SQLYOG … any ideas?

thanks

]]> By: Sylvain261 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-773610 Sylvain261 Wed, 08 Sep 2010 13:13:47 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-773610 Me again. I think that a good workaround might be to change the replication mode for the session issuing the SELECT INTO. SET TRANSACTION ISOLATION LEVEL READ COMMITTED; SET binlog_format = MIXED; INSERT INTO ....SELECT ... FROM .... With this two settings : - There is no lock on the selected tables - The replication of the insert into select statement is done with row base replication witch is save in this isolation level (changing the replication mode is done only for the active session, it should not create application issues). I've not tested it yet. Me again.
I think that a good workaround might be to change the replication mode for the session issuing the SELECT INTO.
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET binlog_format = MIXED;
INSERT INTO ….SELECT … FROM ….
With this two settings :
– There is no lock on the selected tables
– The replication of the insert into select statement is done with row base replication witch is save in this isolation level (changing the replication mode is done only for the active session, it should not create application issues).

I’ve not tested it yet.

]]> By: Sylvain261 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-773604 Sylvain261 Wed, 08 Sep 2010 11:43:38 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-773604 Answer to myself : You are wrong ! I've got the exception : Transaction level 'READ-COMMITTED' in InnoDB is not safe for binlog mode 'STATEMENT'(INSERT INTO XXX SELECT YYY...) It seems that, it is not permitted to issue statement that might break consistency between master and slave. So...In statement replication mode, the OUTFILE workaround seems to be the only one that work if you don't what to set innodb_locks_unsafe_for_binlog to 1. Sorry Peter. Does anyone have an extra workaround ? Thxs. Answer to myself :
You are wrong !

I’ve got the exception : Transaction level ‘READ-COMMITTED’ in InnoDB is not safe for binlog mode ‘STATEMENT’(INSERT INTO XXX SELECT YYY…)

It seems that, it is not permitted to issue statement that might break consistency between master and slave.
So…In statement replication mode, the OUTFILE workaround seems to be the only one that work if you don’t what to set innodb_locks_unsafe_for_binlog to 1.

Sorry Peter.
Does anyone have an extra workaround ?
Thxs.

]]> By: Sylvain261 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-773592 Sylvain261 Wed, 08 Sep 2010 08:49:04 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-773592 Artem, I've made a very simple test (mysql 5.1.45) with 2 concurrent process : Process 1 : INSERT INTO table2 SELECT * FROM table1 ; Process 2 : UPDATE table1 SET myCol= 'X' WHERE myCol= 'XX'; Test case 1 : REPEATABLE-READ => Process 2 is locked until process 1 finish. Test case 2 : READ COMMITTED => Process 2 is not locked So my understanding is that, if you don't mind that your inserted table might be not consistent in your slave with the one in your master : using READ COMMITTED transaction isolation level is the way to go, it's much simpler than the INSERT ... INTO OUTFILE workaround posted by peter. Am I wrong ? Artem,

I’ve made a very simple test (mysql 5.1.45) with 2 concurrent process :

Process 1 :
INSERT INTO table2 SELECT * FROM table1 ;

Process 2 :
UPDATE table1 SET myCol= ‘X’ WHERE myCol= ‘XX’;

Test case 1 : REPEATABLE-READ
=> Process 2 is locked until process 1 finish.

Test case 2 : READ COMMITTED
=> Process 2 is not locked

So my understanding is that, if you don’t mind that your inserted table might be not consistent in your slave with the one in your master : using READ COMMITTED transaction isolation level is the way to go, it’s much simpler than the INSERT … INTO OUTFILE workaround posted by peter.
Am I wrong ?

]]> By: Artem http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/comment-page-1/#comment-768702 Artem Wed, 07 Jul 2010 15:38:44 +0000 http://www.mysqlperformanceblog.com/2006/07/12/insert-into-select-performance-with-innodb-tables/#comment-768702 Has anyone tried changing the isolation level instead of using OUTFILEs. SET TRANSACTION ISOLATION LEVEL READ COMMITTED; INSERT INTO TemporaryTable SELECT ... FROM HighlyContentiousTableInInnoDb; See my stackoverflow question/answer for more info: http://stackoverflow.com/questions/2640898/how-to-improve-insert-into-select-locking-behavior Has anyone tried changing the isolation level instead of using OUTFILEs.

SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
INSERT INTO TemporaryTable SELECT … FROM HighlyContentiousTableInInnoDb;

See my stackoverflow question/answer for more info: http://stackoverflow.com/questions/2640898/how-to-improve-insert-into-select-locking-behavior

]]>