If TC has transaction logging (Berkeley DB has it) you could isolate the tranlog file to a separate “fast” spindle & do an fsync against just that tranlog. The tranlog then plays the actual commits against the main tables.

Another thing to consider is the hotspot in the filesystem (at the end of the data tables or tranlog). Also tuning your I/O buffers & cache to actually be *smaller* might help, avoiding the pitfall of big deferred writes or having to make manual sync/fsync calls from your application.

Likewise if you can push the table’s indices off to a different set of spindle that’s yet more I/O wait averted by hardware-based tuning.

As for indices, if the DBM (TC) allows for using clustered PK indices (esp. with tunable fillfactor) the table itself can be tuned so that write hotspots – in conjunction with any striped RAID – are more likely to spread out across the IO system & reduce IO wait. The table bloats more quickly but the performance gains are readily had.

Basically all tuning tricks for a file-based DBMS are the same as with any RDBMS (like DB/2) apply. The difference is that with a file-baesd DBMS you have to tune the OS more in keeping with a database machine. Think: AS/400 or S390. One can can even schedule such tuning parameters, where the OS buffer/caches might be tuned for certain times of the day/month/year for the a write-intensive periods of heavy writes to flush its dirty pages quickly & regularly in the background. VLDB systems are often tuned accordingly.

Another trick is the appropriate use of RAID. RAID5 brings write CRC overhead (pure striping RAID0 doesn’t). RAID1+0 to the rescue – the hotspots can spread out across a LVM on RAID10 – the writes will be appreciably faster & if you mirror twice the odds are much lower readers will contend on disk heads against a big writer. Although RAID10 brings 2x (or 3x) overhead in terms of disk usage, if it’s reliability & speed you want then hardware to the rescue….

As for speed, the type of IO hardware also plays a role here – SSA is going to be faster & more reliable than SCSI b/c SSA runs on a loop. There’s less I/O contention for starters & a failed drive on a SSA loop won’t disrupt the loop.

Also a good HDD controller would have a cache battery that ensures buffers are flushed if there’s power interruption. The beauty of this is that you can tune your buffers & cache down near the size of the controller’s own cache & know your OS is keeping the data pumped at the pace of the IO controller’s ability to safely perform writes.

This requires understanding what parameters to set in sysctl, but it ain’t rocket science either….

perl does, but using -ext requires a LUA extension.
I’m referencing the latest docs. Did I miss something?

Thanks.

Awesome post and analysis of TC/TT. Mikio wrote his thoughts about this matter on his blog. Thought you’d find it interesting.

http://1978th.net/tech-en/promenade.cgi?id=6

Cheers,
Toru