Comments on: The tool I’ve been waiting for years http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/ Percona's MySQL & InnoDB performance and scalability blog Sat, 11 Feb 2012 16:45:54 +0000 hourly 1 http://wordpress.org/?v=3.2.1 By: Tool update: WTF is inside filesystem cache ? http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-258161 Tool update: WTF is inside filesystem cache ? Thu, 27 Mar 2008 13:04:31 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-258161 [...] inside filesystem cache and tell me what files (or pages of files) are inside. Well Peter Zaitsev points to the fincore tool, which comes pretty damn close: you give it a file, and it will tell you which [...] [...] inside filesystem cache and tell me what files (or pages of files) are inside. Well Peter Zaitsev points to the fincore tool, which comes pretty damn close: you give it a file, and it will tell you which [...]

]]> By: peter http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-256043 peter Sun, 23 Mar 2008 04:32:24 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-256043 Michael, The double buffering has many down sides. Depending on workload and OS it can be more or less. You're right data which is cached is typically removed from MySQL quickly but still. Consider full table scan for example - you get a lot of data which is being read very quickly - consuming a lot of cache quickly it can well remove data which you need. Another issue is battle for virtual memory - a lot of cached IO makes OS to try to make more memory for the cache which often causes it to try swap out something... like some of huge caches you allocated to MySQL. Anyway - this tool is great because now you can actually see how much double buffering you have and what gets cached. Michael,

The double buffering has many down sides. Depending on workload and OS it can be more or less. You’re right data which is cached is typically removed from MySQL quickly but still. Consider full table scan for example – you get a lot of data which is being read very quickly – consuming a lot of cache quickly it can well remove data which you need.

Another issue is battle for virtual memory – a lot of cached IO makes OS to try to make more memory for the cache which often causes it to try swap out something… like some of huge caches you allocated to MySQL.

Anyway – this tool is great because now you can actually see how much double buffering you have and what gets cached.

]]> By: Michael Monashev http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-255842 Michael Monashev Sat, 22 Mar 2008 20:19:47 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-255842 Пётр, спасибо за разъяснение. Я раньше всегда считал, что что двойное кэширование (в MySQL и в OS) - это зло и приводит к тому, что память тратися впустую. А на практике выходит, что в этих двух кэшах содержаться разные данные и память напрасно почти не тратится, ибо данные попавшие в кэш MySQL-я запрашиваются с диска лишь единожды и потом с большой вероятностью вытесняются из кэша OS, т.к. к ним нет запросов на чтение. Пётр, спасибо за разъяснение.

Я раньше всегда считал, что что двойное кэширование (в MySQL и в OS) – это зло и приводит к тому, что память тратися впустую. А на практике выходит, что в этих двух кэшах содержаться разные данные и память напрасно почти не тратится, ибо данные попавшие в кэш MySQL-я запрашиваются с диска лишь единожды и потом с большой вероятностью вытесняются из кэша OS, Ñ‚.к. к ним нет запросов на чтение.

]]> By: peter http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-255712 peter Sat, 22 Mar 2008 17:23:54 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-255712 Thanks. Anyone knows if there is updated version for this too anywhere ? Thanks.

Anyone knows if there is updated version for this too anywhere ?

]]> By: Alex Savchuk http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-255571 Alex Savchuk Sat, 22 Mar 2008 08:27:07 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-255571 Hello, there is an util which looks like what you want: http://oss.oracle.com/projects/codefragments/src/trunk/pagecache-info/pagecache-info _but_ (as usual) there is a problem -- it worked only for 2 old versions of kernel :) And it doesn't work on lastest kernels... Need to update it for supporting new kernels. I was trying but i have small knowledge of kernel :( Hello,

there is an util which looks like what you want:
http://oss.oracle.com/projects/codefragments/src/trunk/pagecache-info/pagecache-info

_but_ (as usual) there is a problem — it worked only for 2 old versions of kernel :)
And it doesn’t work on lastest kernels…

Need to update it for supporting new kernels.
I was trying but i have small knowledge of kernel :(

]]> By: Dathan Vance Pattishall http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-255012 Dathan Vance Pattishall Fri, 21 Mar 2008 04:27:44 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-255012 Wow I always wanted something like this. I had no idea that this even existed. Thank you for letting us know! Wow I always wanted something like this. I had no idea that this even existed. Thank you for letting us know!

]]> By: peter http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-254926 peter Thu, 20 Mar 2008 17:59:25 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-254926 MYD files are expected to be cached more by OS than MYI files because later are cached in key_buffer so OS should get less requests for them and less motivation to have them in the cache. MYD files are expected to be cached more by OS than MYI files because later are cached in key_buffer so OS should get less requests for them and less motivation to have them in the cache.

]]> By: Michael Monashev http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-254868 Michael Monashev Thu, 20 Mar 2008 11:49:08 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-254868 Спасибо большое за интересную утилитку. Узнал много интересного. Например то, что .MYD-файлы кэшируются сильнее .MYI-файлов. Хотя может это только у меня так. Иногда fincore падает почему-то. Но в целом даёт довольно интересную картину. Ещё раз спасибо. Спасибо большое за интересную утилитку. Узнал много интересного. Например то, что .MYD-файлы кэшируются сильнее .MYI-файлов. Хотя может это только у меня так.

Иногда fincore падает почему-то. Но в целом даёт довольно интересную картину. Ещё раз спасибо.

]]> By: peter http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-254669 peter Wed, 19 Mar 2008 21:46:12 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-254669 Yes sure. But the fact the wrapper is in Perl makes it easier to extend for people familiar with scripting language. Hardcore C guys just need to know such call exists :) Yes sure. But the fact the wrapper is in Perl makes it easier to extend for people familiar with scripting language.
Hardcore C guys just need to know such call exists :)

]]> By: Dmitriy T. http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/comment-page-1/#comment-254585 Dmitriy T. Wed, 19 Mar 2008 18:28:02 +0000 http://www.mysqlperformanceblog.com/2008/03/18/the-tool-ive-been-waiting-for-years/#comment-254585 Perl used in this tool just for wrapping C-code, in which mincore called. P.S. Sorry for broken english :) Perl used in this tool just for wrapping C-code, in which mincore called.

P.S. Sorry for broken english :)

]]>