Welcome to the 63rd edition of Log Buffer, the weekly review of database blogs. For those of you reading…

Regarding the 128k optimization it sounds very much like a sync is needed for how the data travels from the harddrive into the cpu. Pretty much similar why a cpu:ram ratio of 1:1 regarding fsb is often better than 1:2 ratio or some other variant.

Jay: Any progress of a tool which can find “optimal values” for a given system?

Peter: I disagree with you regarding example configurations. By using example configuration from a given system you can save much time in order to if not get a perfect configuration at least get a way better configuration than the outdated examples which are included with the mysql itself. It sounds more that you want to protect your consulting business to help others to find their optimal values with such statement as you gave in the end of this article.

Most people involving in security should by now know that security by obscurity doesnt work, I think the same applies in order to find optimal values for different setup. Optimization by obscurity doesnt work either - so lets share our findings and configuration examples

Indeed. However read_buffer_size is used for sequential scans - it is not used if table is read via index lookup for example.

In this case I performed test on virtually empty file system so I would be surprised if file is significantly fragmented.

Also we’re speaking about logical IO to file anyway so if file is fragmented one large read to the file will be split to several smaller physical reads but there would not be more of these for larger buffer.

your theory using larger buffers is valid for sequential IO only. Having random IO large buffers will result in lot of unneccessary read ahead data. And if file system hadn’t written file in consecutive physical segments (fragmentation), then theory becomes much more complicated …

As far as I remember Innodb tables are not read using read_buffer

O_DIRECT does use different I/O path, but that is irrelevant. What is relevant that even InnoDB tables are read by using read buffer and that such combo would dodge out the consequences of OS cache-ing … I think I will try to do myself what I suggested, as soon as I find some time ……… ;o)

The interesting thing however if you’re doing reads larger than stripe size you should get better performance, not worse - aligned
128K read will have to be read from single hard drive while 256K could use both hard drives for parallel IO.

As we go to higher block sizes in theory RAID should be able to do even more optimization - say you do 2MB aligned read with 128M size - RAID could perform 1M read for each of underlying devices and then reorder 128K blocks to give requested 2M result set giving close to twice of sequential read speed from single device (assuming there is enough bus bandwidth for it)

When I have some time I’ll try to rerun this test on my Dell PowerEdge 2950 box which has more hard drives and so in theory must love IO in large blocks.