For about the past year I’ve been formulating a series of tools and practices that can provide deep insight into system performance simply by looking at TCP packet headers, and when they arrive and depart from a system. This works for MySQL as well as a lot of other types of systems, because it doesn’t require any of the contents of the packet. Thus, it works without knowledge of what the server and client are conversing about. Packet headers contain only information that’s usually regarded as non-sensitive (IP address, port, TCP flags, etc), so it’s also very easy to get access to this data even in highly secure environments.
I’ve finally written up a paper that shows some of my techniques for detecting problems in a system, which can be an easy way to answer questions such as “is there something we should look into more deeply?” without launching a full-blown analysis project first. It’s available from the white paper section of our website: MySQL Performance Analysis with Percona Toolkit and TCP/IP Network Traffic
Baron
Thanks for the white paper. I think this is definitely a useful tool to replicate prod load for benchmarking systems.
Great paper, Baron. I am a big fan of pt-tcp-model. It’s very versatile and because it is protocol agnostic, it can be used to analyze all sorts of different technologies including memcached, http, redis, mongodb, etc. I used it recently to help diagnose an upstream packet delivery issue – http://blog.9minutesnooze.com/analyzing-http-traffic-tcpdump-perconas-pttcpmodel/