With MyISAM there is no such thing as “won’t loose any data” if server crashes some MyISAM updates can be only in OS cache which will be lost furthermore tables can become corrupted.

This is all probability thing of course.

Let me answer your questions one by one

1) Your understanding about flushing is aproximately correct. The data is modified directly in the buffer pool pages, which becomes “dirty” if there are some unflushed changes and later these pages will need to be flushed in full. This actually happens via one more “physical” log file which is called “double write buffer” to handle problem of partial page writes. The log file is structured differently it stores records which apply to individual page modification operation perfomed rather than full pages.

Unless database is staying idle database on the disk is pretty much always out of sync. Only in case of shutdown they are synchronozed (this is why it can take a while). They are also synchronized by crash recovery.

2) Right in MySQL binary log file has to be XA source as well… otherwise it can run out of sync. “Traditional” databases would not normally have many storage engines each with their own logs. In MySQL 5.0 you can disable innodb_xa_support and you can get binary log possibly out of sync with innodb transactional log. However it will not make group commit to work. Group commit only works in MySQL 5.0 if binary log is disabled at all.

Take a look at these benchmarks: http://www.mysqlperformanceblog.com/2006/05/19/group-commit-and-xa/ – disabling XA saves you from extra fsync which allows to double results but results still do not improve with many concurrent threads. This would be sight of good working group commit.

Also, as you mentioned the issue of group commit being broken (see http://www.mysqlperformanceblog.com/2006/05/19/group-commit-and-xa), I read that entry with interest, and what I took from it was that you should disable xa-support in my.cnf if you
1. don’t need it and
2. have binary logging (replication) enabled
your benchmarks suggested that by doing that (even if group commit is “broken”) you can double your write throughput

However, i stumbled upon this in the official mySQL 5.0 documentation:
“Even with sync_binlog set to 1, there is still the chance of an inconsistency between the table content and binary log content in case of a crash. For example, if you are using InnoDB tables and the MySQL server processes a COMMIT statement, it writes the whole transaction to the binary log and then commits this transaction into InnoDB. If the server crashes between those two operations, the transaction is rolled back by InnoDB at restart but still exists in the binary log. This problem can be solved with the –innodb-safe-binlog option, which adds consistency between the content of InnoDB tables and the binary log. (Note: –innodb-safe-binlog is unneeded as of MySQL 5.0; it was made obsolete by the introduction of XA transaction support.)”

I’m not sure what to make of that. XA support is traditionally for 2-phase commits between two different databases (or other transactional resources), i wouldn’t think that the binary log ITSELF is XA-compliant:)

I guess what I’m really asking is, if you have enabled binary logging (replication), sync_binlog = 1, and you’re using MySQL 5, and it’s the only transactional resource you’re using, is it safe to DISABLE innodb_xa_support? i.e. will the potential extra “dirty” statement remain in the replication log? I guess I don’t understand how enabling xa-support would resolve this, and as your benchmarks indicate, it potentially makes a HUGE difference (group commit or not).
Sorry for the length of this post, but I think you can see it’s a liitle bit confusing:)

Thanks

Also, as you mentioned the issue of group commit being broken (see

Then at some later time, as part of a background process, write the “dirty” pages in the buffer pool to the .ibd files
I’m sure I’m leaving some things out, but is that essentially the sequence? I had assumed that the innodb-log-files were “flushed” at some point to the tablespace files, just because in case of a crash before all buffered dirty pages were flushed, then InnoDB has to use its log files to synchronize the tablespace files because obviously anything in memory would have been lost, but it seems like you’re suggesting that the log files are not used to synchronize the tablespace files EXCEPT in the case of a crash?

Also, as you mentioned the issue of group commit being broken (see

Yes you’re right. Binary log is protected by mutex which makes it very expensive to do sync_binlog=1 unless you have battery backed up cache and if your OS does not lie you about fsyncs. It is simple code which does not even have “group commit” allowing to flush several events with single write to disk. Yes this means log writes are going to be serialized. It is not exactly single threaded as flushing the log is probably only portion of what thread does but it can be significant.

Innodb also has log mutex which protects log writes. In MySQL 4.1 and MySQL 5.0 without binary logging it however can do group commit, meaning it can commit several transactions which are waiting to be commited at the same time with single log write. This is currently broken in 5.0 if you enable binary log.

In general data migration in Innodb happens as following (which is traditional way for most of database management systems) – once transaction is commited the data is recorded in Innodb log files before COMMIT returns. The data pages are however modified only in the buffer pool. Dirty pages are flushed from the buffer pool in background. For Innodb flushes happen when log files are almost full and Innodb needs to overwrite some old log records (remember log files in Innodb are circular) or if there are too many dirty buffers in the buffer pool (see innodb_max_dirty_pages_pct variable).