Have a look at what we are doing with Tungsten Replicator (http://www.continuent.com/community/tungsten-replicator). One of the problems with multiple slaves is that you have difficulty promoting a slave to master because the other slaves are holding references to the original files in the old master’s binlog directory. We use global replication IDs which solves that problem. There are other solutions mentioned above (like DRBD); however, I think the slave promotion problem is a substantial reason for the popularity of MMM.

Cheers, Robert

Usually Supermicro boxes are just commodity hardware, you don’t have to use their boards with the chassis. Also the choice of RAID Controller is usually yours to make (which is nice, given that there are at least some vendors who ship *decent* CLI tools for linux like 3ware, not just drivers). I don’t buy directly from supermicro though, I don’t know if you can even get a complete server (loaded with CPUs, disks, RAM and so on) from them. The reason to buy IBM, Dell, Sun or HP (forgot anyone?) is usually Service Level Agreements, you’ll always have the same guys to yell at when something breaks (be it disk, fan, PSU, whatever)

I pulled a number of supermicro servers out of production that were running fedora and i tried to replace with CentOS 4 and 5 and found that it was extremely hard to make the raid controllers on most of these machines work unlike the IBM machines. I’m not 100% sure how fedora was even installed on these machines, best guess is someone spent hours trying all kinds of experaments with drivers disk before find something that worked.

Also i love being able to download all the firmware and drivers for the machines from one place and being able to punch in a machine type into a search box and get everything i need for the machine. If you want a server that holds lots of disk have a look at the sun storage servers that have 50 disks in them. I’m not a big fan of sun hardware due to some bad experiences with Sun V20 servers, but i like these machines. Were looking into deploying a vmware esxi solution for our App servers with NAS storage and these would be a great idea.

Thanks a lot for the information, appreciate it. Have found distributors for the Supermicro machines. It’s pretty amazing to see this 24 drives machine.

Peter: “if you use LVM you can also get another shelf of drives and so expand storage.” – but limited to local drives or external SCSI, right?

Regarding growth – if you use LVM you can also get another shelf of drives and so expand storage.
Regarding drives the 8 is the classic limit these days though many offer external directly attached storage which is more expensive than just internal drives but far cheaper than SAN.

What Baron tells about Drives is true. Check this out:
http://www.supermicro.com/products/chassis/2U/216/SC216A-R900U.cfm

This is 2U chassis from the inexpensive vendor which offers 24 drives

I’m comparing the same number of drives. Note if you have SAN with 200 drives you will likely have multiple servers compared to it which all will share resources. For example it could be compared to 10 servers with shelf of 20 drives compared to it. Indeed because the servers may have spikes at different types there is the potential for the gains. With a lot of Sharded applications with MySQL the load is however very ballanced and predictable with spikes well aligned so gain is less. Also note the complication with performance analisis – Virtual volume is not predictable – sometimes you have performance of 200 drives dedicated to you and sometimes you may end up competing with 20 applications for the same.

Now regarding write acknowledgment. Indeed you get no more than 1ms for SAN while you get something like 50 microseconds for the BBU on the local RAID – you just have PCI Express to reach the cache which is a bit shorter.

And again you mention the cost here quite correctly. My point is SAN is rarely a path to cost efficient performance.

@Baron,
I always wondered why they don’t design our normal family vehicles after cool racing cars. It’s just the bending… Everything in our world revolves around that, I suppose…
Good to head about ’supermicro’. Will look them up. Need to check if they bring these to my country.

Shlomi

The limitations on local disks are usually because the manufacturers want to limit you, not because it’s impossible to do. They have higher margins on SAN and other external storage devices which are not commodity items. But some manufacturers (supermicro, for example) will let you put lots of disks inside the chassis itself.

sure, OS being on separate disks will not hurt, all I say – they could help a lot if MySQL had more disks to operate on i.e. 6 rather than 4 in the case with 6 slots you mentioned with OS sleeping on 5G in the corner anyway. Also, I’m not sure I understand reasoning behind using smallest disks for OS – except for the price of course, though difference ain’t significant these days.

Regarding “OS is above all else and should always function” – on a dedicated MySQL server I see no big difference whether OS (and MySQL) is down or only MySQL in case of say 2-disk failure in RAID-5 – in both cases MySQL service is not available and you have to physically fix the server anyway (or do some trick over KVM if it is fixable). Filesystem corruption is different case, but sure I agree MySQL is better off on different file system than OS.

Disk space needs indeed can be very dynamic, but quite often disk space and other resource needs for MySQL-backed applications correlate while scaling and I happen to find server needs more memory or CPUs [than it can accommodate] sooner. This is just my experience and it varies a lot from application to application – indeed “you never know”

Cheers!