Comments on: How fast can you sort data with MySQL ? http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/ Percona's MySQL & InnoDB performance and scalability blog Sat, 11 Feb 2012 16:45:54 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Avi http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-773596 Avi Wed, 08 Sep 2010 09:32:44 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-773596 Hi Is there any way that I can sort in numeric mode or in string mode.....????? Let me know any if possible. Thanks Hi
Is there any way that I can sort in numeric mode or in string mode…..?????
Let me know any if possible.
Thanks

]]> By: pascal http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-769951 pascal Sat, 24 Jul 2010 08:04:05 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-769951 I second other commenters asking for commented My.cnf files from the magicians of this blog! that would certainly be very interesting and useful. I second other commenters asking for commented My.cnf files from the magicians of this blog! that would certainly be very interesting and useful.

]]> By: michael.ma的奋斗人生 » Blog Archive » （转贴）根据status信息对MySQL服务器进行优化 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-625576 michael.ma的奋斗人生 » Blog Archive » （转贴）根据status信息对MySQL服务器进行优化 Fri, 07 Aug 2009 06:39:56 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-625576 [...] Sort_merge_passes 包括两步。MySQL 首先会尝试在内存中做排序，使用的内存大小由系统变量 Sort_buffer_size 决定，如果它的大小不够把所有的记录都读到内存中，MySQL 就会把每次在内存中排序的结果存到临时文件中，等 MySQL 找到所有记录之后，再把临时文件中的记录做一次排序。这再次排序就会增加 Sort_merge_passes。实际上，MySQL 会用另一个临时文件来存再次排序的结果，所以通常会看到 Sort_merge_passes 增加的数值是建临时文件数的两倍。因为用到了临时文件，所以速度可能会比较慢，增加 Sort_buffer_size 会减少 Sort_merge_passes å’Œ 创建临时文件的次数。但盲目的增加 Sort_buffer_size 并不一定能提高速度，见 How fast can you sort data with MySQL?（引自http://qroom.blogspot.com/2007/09/mysql-select-sort.html，貌似被墙） [...] [...] Sort_merge_passes 包括两步。MySQL 首先会尝试在内存中做排序，使用的内存大小由系统变量 Sort_buffer_size 决定，如果它的大小不够把所有的记录都读到内存中，MySQL 就会把每次在内存中排序的结果存到临时文件中，等 MySQL 找到所有记录之后，再把临时文件中的记录做一次排序。这再次排序就会增加 Sort_merge_passes。实际上，MySQL 会用另一个临时文件来存再次排序的结果，所以通常会看到 Sort_merge_passes 增加的数值是建临时文件数的两倍。因为用到了临时文件，所以速度可能会比较慢，增加 Sort_buffer_size 会减少 Sort_merge_passes å’Œ 创建临时文件的次数。但盲目的增加 Sort_buffer_size 并不一定能提高速度，见 How fast can you sort data with MySQL?（引自http://qroom.blogspot.com/2007/09/mysql-select-sort.html，貌似被墙） [...]

]]> By: 根据status信息对MySQL服务器进行优化（二） » 超群.com的博客 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-518208 根据status信息对MySQL服务器进行优化（二） » 超群.com的博客 Wed, 25 Mar 2009 07:08:29 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-518208 [...] Sort_merge_passes 包括两步。MySQL 首先会尝试在内存中做排序，使用的内存大小由系统变量 Sort_buffer_size 决定，如果它的大小不够把所有的记录都读到内存中，MySQL 就会把每次在内存中排序的结果存到临时文件中，等 MySQL 找到所有记录之后，再把临时文件中的记录做一次排序。这再次排序就会增加 Sort_merge_passes。实际上，MySQL 会用另一个临时文件来存再次排序的结果，所以通常会看到 Sort_merge_passes 增加的数值是建临时文件数的两倍。因为用到了临时文件，所以速度可能会比较慢，增加 Sort_buffer_size 会减少 Sort_merge_passes å’Œ 创建临时文件的次数。但盲目的增加 Sort_buffer_size 并不一定能提高速度，见 How fast can you sort data with MySQL?（引自http://qroom.blogspot.com/2007/09/mysql-select-sort.html，貌似被墙） [...] [...] Sort_merge_passes 包括两步。MySQL 首先会尝试在内存中做排序，使用的内存大小由系统变量 Sort_buffer_size 决定，如果它的大小不够把所有的记录都读到内存中，MySQL 就会把每次在内存中排序的结果存到临时文件中，等 MySQL 找到所有记录之后，再把临时文件中的记录做一次排序。这再次排序就会增加 Sort_merge_passes。实际上，MySQL 会用另一个临时文件来存再次排序的结果，所以通常会看到 Sort_merge_passes 增加的数值是建临时文件数的两倍。因为用到了临时文件，所以速度可能会比较慢，增加 Sort_buffer_size 会减少 Sort_merge_passes å’Œ 创建临时文件的次数。但盲目的增加 Sort_buffer_size 并不一定能提高速度，见 How fast can you sort data with MySQL?（引自http://qroom.blogspot.com/2007/09/mysql-select-sort.html，貌似被墙） [...]

]]> By: Project 2061 Techlog » Optimizing MySQL Server Runtime Parameters http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-296421 Project 2061 Techlog » Optimizing MySQL Server Runtime Parameters Wed, 07 May 2008 18:55:24 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-296421 [...] MySQL Performance Blog: How fast can you sort data with MySQL? [...] [...] MySQL Performance Blog: How fast can you sort data with MySQL? [...]

]]> By: Justin http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-171970 Justin Tue, 25 Sep 2007 05:31:49 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-171970 Further to my last question.. As described in this blog post, a table scan with order by/limit does a very fast sort if you want to get the top-most rows .. but I find it useless if you want to use it to create a full ordered insert (for example) for innodb. My large table spends hours 'sorting result', even though I can get top 20 results within 15 minutes. I wonder if there are some short-cuts taken by mysql where limit is used. Perhaps a table-scan, with a running best-of list, for example. That would not then be a sort, would it? In order to do an ordered bulk insert to innodb by PK I am resorting to external sort of a file with unix sort then feeding it with batch inserts, rather than feeding it to myisam then converting, or feeding to myisam then sort .. order by .. insert, or feeding un-ordered PK to innodb directly then doing alter table engine=innodb to "defragment" PK. Am I on the right track? Further to my last question..

As described in this blog post, a table scan with order by/limit does a very fast sort if you want to get the top-most rows .. but I find it useless if you want to use it to create a full ordered insert (for example) for innodb.
My large table spends hours ‘sorting result’, even though I can get top 20 results within 15 minutes. I wonder if there are some short-cuts taken by mysql where limit is used. Perhaps a table-scan, with a running best-of list, for example. That would not then be a sort, would it?

In order to do an ordered bulk insert to innodb by PK I am resorting to external sort of a file with unix sort then feeding it with batch inserts, rather than feeding it to myisam then converting, or feeding to myisam then sort .. order by .. insert, or feeding un-ordered PK to innodb directly then doing alter table engine=innodb to “defragment” PK.

Am I on the right track?

]]> By: Justin http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-171897 Justin Tue, 25 Sep 2007 02:04:36 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-171897 On this subject.. if you have a huge un-ordered myisam table and wish to get it into sort order before dumping it to INNODB, which is going to be faster? a) Add an index on column to be sorted and select by that index b) Dump the data to flat file and sort using unix sort then re-import c) Select using table-scan and sort as per your example d) alter to innodb with primary key (which is the objective anyway) I'm currently doing (a) but I'm wondering if this is a big waste of time. On this subject.. if you have a huge un-ordered myisam table and wish to get it into sort order before dumping it to INNODB, which is going to be faster?

a) Add an index on column to be sorted and select by that index
b) Dump the data to flat file and sort using unix sort then re-import
c) Select using table-scan and sort as per your example
d) alter to innodb with primary key (which is the objective anyway)

I’m currently doing (a) but I’m wondering if this is a big waste of time.

]]> By: peter http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-169796 peter Wed, 19 Sep 2007 10:20:31 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-169796 Lachlan, Indeed it can affect things. And there are more variables like table structure of course - I have not tested these yet. Lachlan,

Indeed it can affect things. And there are more variables like table structure of course – I have not tested these yet.

]]> By: Lachlan Mulcahy http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-169703 Lachlan Mulcahy Wed, 19 Sep 2007 05:40:04 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-169703 Hi Peter, I wonder how the size of this buffer would affect real loaded systems? Obviously on many real systems disk IO comes into the equation and then disk seek time can start playing a big factor. I would be interested to see if the results change in this kind of situation. Kind Regards, Lachlan Hi Peter,

I wonder how the size of this buffer would affect real loaded systems?

Obviously on many real systems disk IO comes into the equation and then disk seek time can start playing a big factor.

I would be interested to see if the results change in this kind of situation.

Kind Regards,
Lachlan

]]> By: MySQL Performance Blog » MySQL: what read_buffer_size value is optimal ? http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/comment-page-1/#comment-169054 MySQL Performance Blog » MySQL: what read_buffer_size value is optimal ? Mon, 17 Sep 2007 14:41:51 +0000 http://www.mysqlperformanceblog.com/2007/08/18/how-fast-can-you-sort-data-with-mysql/#comment-169054 [...] benchmarks and performance research. I just recently wrote about rather surprising results with sort performance and today I've discovered even read_buffer_size selection may be less than [...] [...] benchmarks and performance research. I just recently wrote about rather surprising results with sort performance and today I’ve discovered even read_buffer_size selection may be less than [...]

]]>