Comments on: Guidance for MySQL Optimizer Developers http://www.mysqlperformanceblog.com/2009/09/20/guidance-for-mysql-optimizer-developers/ Everything about MySQL Performance Sun, 21 Mar 2010 02:39:05 +0000 hourly 1 By: peter http://www.mysqlperformanceblog.com/2009/09/20/guidance-for-mysql-optimizer-developers/comment-page-1/#comment-657333 peter Thu, 24 Sep 2009 18:50:17 +0000 http://www.mysqlperformanceblog.com/?p=1211#comment-657333 Clint, I think Drizzle is moving in right direction with this. And NDB (MySQL Cluster) is another example You should not be forced to ONLY access your storage using SQL. There are cases when you can avoid expense of parsing and picking optimizer plan when transaction is predictable and performance critical. Clint,

I think Drizzle is moving in right direction with this. And NDB (MySQL Cluster) is another example
You should not be forced to ONLY access your storage using SQL. There are cases when you can avoid expense of parsing and picking optimizer plan when transaction is predictable and performance critical.

]]> By: Clint Byrum http://www.mysqlperformanceblog.com/2009/09/20/guidance-for-mysql-optimizer-developers/comment-page-1/#comment-657262 Clint Byrum Thu, 24 Sep 2009 16:35:16 +0000 http://www.mysqlperformanceblog.com/?p=1211#comment-657262 We've all been bit by the optimizer suddenly picking a poor plan when the data set changes, most probably without even knowing that the problem lies therein. Its not like saving an explain of every query is really all that common practice (though we have to give a nod to mk-query-digest which would make doing that relatively easy). The most common scenario is probably "hey query X is going REALLY slow on live but its really fast on dev.. explain both.. realize that query on live is now scanning a table.. add index hints or archive some data. This is all reasonable, but it probably means angry database users waiting on highly technical work to complete, which is never a good place to be in. I think this is a large driving force behind people moving bits of their architecture to simpler key/value systems. Among other things, the value of predictable scalability is outweighing the flexibility of SQL, even when it means rewriting and refactoring large pieces of applications. So yes, I have to agree that adding more places to force queries to run with a predictable plan is indeed something MySQL will need to continue to meet the demands of users. We’ve all been bit by the optimizer suddenly picking a poor plan when the data set changes, most probably without even knowing that the problem lies therein. Its not like saving an explain of every query is really all that common practice (though we have to give a nod to mk-query-digest which would make doing that relatively easy). The most common scenario is probably “hey query X is going REALLY slow on live but its really fast on dev.. explain both.. realize that query on live is now scanning a table.. add index hints or archive some data. This is all reasonable, but it probably means angry database users waiting on highly technical work to complete, which is never a good place to be in.

I think this is a large driving force behind people moving bits of their architecture to simpler key/value systems. Among other things, the value of predictable scalability is outweighing the flexibility of SQL, even when it means rewriting and refactoring large pieces of applications.

So yes, I have to agree that adding more places to force queries to run with a predictable plan is indeed something MySQL will need to continue to meet the demands of users.

]]> By: peter http://www.mysqlperformanceblog.com/2009/09/20/guidance-for-mysql-optimizer-developers/comment-page-1/#comment-656785 peter Wed, 23 Sep 2009 20:54:52 +0000 http://www.mysqlperformanceblog.com/?p=1211#comment-656785 Didier, There are a lot of cases why optimizer can do mistakes. I have not seen optimizer which would never pick the slower plan for some kind of query. It is good if optimizer is fixed but you can't always upgrade to the next release. For example if you have problem with MySQL 5.0 the problem may be fixed in 5.4.x but you may not really be able to upgrade immediately. Didier,

There are a lot of cases why optimizer can do mistakes. I have not seen optimizer which would never pick the slower plan for some kind of query. It is good if optimizer is fixed but you can’t always upgrade to the next release. For example if you have problem with MySQL 5.0 the problem may be fixed in 5.4.x but you may not really be able to upgrade immediately.

]]> By: Didier Spezia http://www.mysqlperformanceblog.com/2009/09/20/guidance-for-mysql-optimizer-developers/comment-page-1/#comment-656767 Didier Spezia Wed, 23 Sep 2009 19:52:38 +0000 http://www.mysqlperformanceblog.com/?p=1211#comment-656767 I cannot agree more. Some database projects (for instance PostgreSQL) have the following policy: if the optimizer is wrong, then the optimizer must be fixed, and there is no point supporting hints. Rationale is this policy is supposed to increase the quality of the optimized in the long term. It may work or not work, this is debatable. I will not comment on the relative quality of Oracle/MySQL/PostgreSQL query optimizer. My point is it is not really a pragmatic approach. Giving full control over the query execution plans to the developer or DBA is an invaluable tool to get the job done, especially for large databases, in a high throughput, high availability context. Results of a query optimizer can only be as good as the collected statistics, which are sometimes quite difficult to generate in some situations. I cannot agree more. Some database projects (for instance PostgreSQL) have the following policy: if the optimizer is wrong, then the optimizer must be fixed, and there is no point supporting hints. Rationale is this policy is supposed to increase the quality of the optimized in the long term. It may work or not work, this is debatable. I will not comment on the relative quality of Oracle/MySQL/PostgreSQL query optimizer.

My point is it is not really a pragmatic approach. Giving full control over the query execution plans to the developer or DBA is an invaluable tool to get the job done, especially for large databases, in a high throughput, high availability context. Results of a query optimizer can only be as good as the collected statistics, which are sometimes quite difficult to generate in some situations.

]]>