I’m confused by what I’ve read: After a crash, the slave is now live. How long does it take to resync the master with the slave? Can the slave simply become the master and the master now be considered the hot spare slave that will take over if the new “master” machine ever crashes?

Thanks,

Any comments (& the sooner the better) would be helpful, also any direction to a community of folks who are deeply involved with these technologies would be helpful as well.

- Joel

Again my point is not to say generally it is good or bad but highlight the problems you can run into

I do not think it is as easy as you say. you can limit number of io for REDO but there is also UNDO part which depends only on size of transaction. It is indeed background in 5.0 but the data remains locked until rollback is complete which can affect operation.

Plus you still have warmup problem.

Speaking about Google batch - I think there is a lot of cool stuff out where besides HA, and thanks for explanations.

Is there any parallelism or concurrency (multiple threads or multiple IOs pending) in InnoDB crash recovery code?

The Google patch had more than semi-sync for an HA solution with MySQL. It also includes a feature to keep the InnoDB transaction log and slave replication state in sync after a crash (transactional replication). But the most interesting feature was mirror binlog. This maintained mirrors of the master’s binlog on a slave (same filename as used on the master, byte for byte equivalent). This was done using the same replication protocol — the IO thread would copy events into the relay log and the mirrored binlog. It allows hierarchical replication and transparent slave failover.

We are trying to redo support for hierarchical replication and transparent slave failover with global transaction IDs.

Сам не пробовал такую схему реализовывать, но возможно она окажется быстрее по чтению. Особенно по случайному.

Another thing I’d like to add is that there are some database/application designs that just don’t allow for friendly multi-master setups. More real-time data warehouse applications that log tons of data and frequently summarize and aggregate will run into problems. Do you run aggregation scripts only on one server and propagate them? Do you run aggregates on both servers and “assume” that all the numbers add up in the end - this can be very dangerous.

Also, merely having a slave sitting ready to take master can be troublesome in some situations because the slave is still single-threaded. You end up getting a beefier slave than your production system just so that a single thread on the slave can keep up with all your transactional threads on your master. Now you have to ask yourself - maybe I can use the slave for other things, so it doesn’t bother me too much that my slave is actually more powerful than my master. Or maybe it’s better to have two equally setup systems and use DRBD instead.

I think given the argument that we all agree on one thing - it really depends on your application/database environment, your requirements for recovery speed, data loss, etc.

Finally, I will leave with this note - we actually do both - drbd + ha cluster for high-availability and a big beefy slave for redundancy and read only queries.

I would add another point: Backup from slave / secondary. With DRBD this is hard / impossible in many situations.

We use both, replication and DRBD in production. One database which is doing nothing but “spooling” working sets runs on DRBD (no slave lag, 100% sure all transactions are there, no need to backup this database). The main database is configured in Master-Master replication fashion. We evaluated using DRBD here too but did not do it because we’d loose the ability to do “online schema change” or LVM snapshot backup. And finally we were not able to get a descent InnoDB recovery time (26G buffer pool). So it would not even be a cold stand-by but a frozen one.

So all the pros and cons stroke us.

As I discussed at the conference (http://www.primebase.org/download/pbxt-uc-2008.pdf), I am planning to add HA capabilities to PBXT using synchronous replication at the engine level.

What do you think of this approach?

Best regards,

Paul