Comments on: How much overhead is caused by on disk temporary tables

By: Dan’s Blog » Blog Archive » Copying to tmp table

Dan’s Blog » Blog Archive » Copying to tmp table — Mon, 26 Jan 2009 16:25:42 +0000

[...] For more information, see this blog. [...]

By: fdask

fdask — Wed, 14 Jan 2009 14:44:36 +0000

Great article here. A lot of MySQL optimization articles don’t even bother looking at disk i/o, so that was a refreshing change. Also the ORDER BY NULL was something I never knew about! Cheers.

By: Wow, never would’ve guessed this. at ConvolutedTheory

Wow, never would’ve guessed this. at ConvolutedTheory — Sun, 05 Oct 2008 06:09:38 +0000

[...] would have guessed that on-disk temporary tables are that bad for performance. Peter over at the MySQL Performance Blog shows otherwise. Share and [...]

By: links for 2008-09-15

links for 2008-09-15 — Mon, 15 Sep 2008 21:31:19 +0000

[...] How much overhead is caused by on disk temporary tables | MySQL Performance Blog (tags: temporary tables performance overhead mysql disk development tuning) Interesting Links [...]

By: vijay

vijay — Thu, 11 Sep 2008 10:03:17 +0000

In my webpage, I want to get a output in a XSL file. The data are retrieved from Mysql db. Retrieving the bulk records in a single file. When I access that download file page simultaneously multi systems in the remote, I couldn’t get the file also my data are crashed. What is the problem? PHP.ini or DB? Plz tell me clearly. And how to solve this problem?

Urgent. Thanks in advance.

By: vijay

vijay — Thu, 11 Sep 2008 10:00:49 +0000

When i retrieve the data from mysql tables using PHP, the output is in a single XSL file. if my web page accessing for download the xsl file many users simultaneously, the little data retrieved . but many rows in my tables using joins. what is the problem? any memory problem or key buffer limit problem, plz let me know as soon as possible, im in hurry.

thanking u

By: krteQ

krteQ — Wed, 30 Apr 2008 11:40:50 +0000

Hi!

I’m using a tmpdir on the tmpfs for about 14 days and it performs very well (5.0.51a). The only problem I have run into is creating temporary InnoDB table (file per table setting), because tmpfs cannot do IO_DIRECT. It prints a warning to the error log every time I create a tmp innoDB table. However, despite these warnings, I think the result is OK and the table is created (see http://bugs.mysql.com/bug.php?id=26662)

krteQ

By: peter

peter — Mon, 21 Apr 2008 15:55:01 +0000

Right. I should again check tmpfs vs heap table to see how it works.

In general of course it would be great if MySQL would have the option to use tmpfs only up to certain space and when switch to the drive – this would allow to use it safer.

Though I guess it is going in different direction – there is a patch from Ebay to support BLOB/TEXT in memory tables and there is Maria comming which will cache the data in pagecache avoiding disk IO for temp tables if they are small enough.

By: müzso

müzso — Mon, 21 Apr 2008 11:02:19 +0000

The MySQL bug with tmpdir was fixed in v5.0.48 (if somebody didn’t know). The current (at the time of writing) stable Debian package has v5.0.32 so if you’re using tmpfs for MySQL’s tmpdir, keep an eye out for the Debian upgrades of the MySQL package (it might be possible that an upgrade will take you to a MySQL server version that has this bug).

Btw. using tmpfs for tmpdir helped me a lot. Our server (used for a news portal) was totally overloaded with I/O when MySQL started to write out several n*100MB temp tables to disk (a tmp table was mostly less than 200MB, but several tmp tables were created in a row resulting in heavy I/O). Sometimes this took a minute or so and the portal came down to it’s knees (eg. it slowed down and started to give “Internal server error” messages in the end). Using a 500MB tmpfs fixed this and everything seems to be just fine since then. However this is just a quick fix, because those large tmp tables should not even be created in the first place.

By: peter

peter — Thu, 28 Feb 2008 06:06:18 +0000

Clint,

Note the bug I described in the blog post was only added in 5.0.44 and fixed promptly in next versions so I just was not too lucky. In general using tmpfs indeed helps a lot especially for moderately sized temporary tables which fit in the cache.

Another interesting development is Maria which would be very good for temporary tables as it caches data not only indexes.