May 25, 2013

Distributed Set Processing with Shard-Query

Can Shard-Query scale to 20 nodes? Peter asked this question in comments to to my previous Shard-Query benchmark. Actually he asked if it could scale to 50, but testing 20 was all I could due to to EC2 and time limits. I think the results at 20 nodes are very useful to understand the performance: [...]

Is your server’s performance about to degrade?

I’ve been talking and writing a bit lately about the scaling problems I’m seeing on fast servers running lots of queries. As a rough guide, I’m seeing this in servers running 20k queries per second and higher, lots of memory, lots of CPU cores, and most queries are running faster than one millisecond; some in [...]

The tool I’ve been waiting for years

I’ve just been pointed to the nice tool which I was waiting for years to see. It is fincore – little perl script which allows you to see what pages of file are cached in OS memory. This is really cool.