June 23, 2008

Neat tricks for the MySQL command-line pager

Posted by Baron Schwartz

How many of you use the mysql command-line client?  And did you know about the pager command you can give it?  It's pretty useful.  It tells mysql to pipe the output of your commands through the specified program before displaying it to you.

Here's the most basic thing I can think of to do with it: use it as a pager.  (It's scary how predictable I am sometimes, isn't it?)

PLAIN TEXT
SQL:
  1. mysql> pager less
  2. mysql> SHOW innodb STATUS\G

For big result sets, it's a pretty handy way to be able to search and scroll through. No mouse required, of course.

But it doesn't have to be this simple! You can specify anything you want as a pager. Hmm, you know what that means? It means you can write your own script and push the output through it. You can't specify arguments to the script, but since you can write your own, that's not really a limitation.(Edit: I'm wrong! You can. See Giuseppe's comment below.) For example, here's a super-simple script that will show the lock waits in the output of SHOW INNODB STATUS. Save this file as /tmp/lock_waits and make it executable.

PLAIN TEXT
CODE:
  1. #!/bin/sh
  2.  
  3. grep -A 1 'TRX HAS BEEN WAITING'

Now in your mysql session, set /tmp/lock_waits as your pager and let's see if there are any lock waits:

PLAIN TEXT
SQL:
  1. mysql> pager /tmp/lock_waits
  2. PAGER SET TO '/tmp/lock_waits'
  3. mysql> SHOW innodb STATUS\G
  4. ------- TRX HAS BEEN WAITING 50 SEC FOR THIS LOCK TO BE GRANTED:
  5. RECORD LOCKS space id 0 page no 52 n bits 72 INDEX `GEN_CLUST_INDEX` of TABLE `test/t` trx id 0 14615 lock_mode X waiting
  6. 1 row IN SET, 1 warning (0.00 sec)

Pretty useful, isn't it? But we can do even more. For example, the Maatkit tools are specifically designed to be useful at the command line in the traditional Unix pipe-and-filter manner. What sort of goodies can we think of here?

PLAIN TEXT
SQL:
  1. mysql> pager mk-visual-EXPLAIN
  2. PAGER SET TO 'mk-visual-explain'
  3. mysql> EXPLAIN SELECT * FROM sakila.film INNER JOIN sakila.film_actor USING(film_id) INNER JOIN sakila.actor USING(actor_id);
  4. JOIN
  5. +- Bookmark lookup
  6. |  +- TABLE
  7. |  |  TABLE          actor
  8. |  |  possible_keys  PRIMARY
  9. |  +- UNIQUE INDEX lookup
  10. |     KEY            actor->PRIMARY
  11. |     possible_keys  PRIMARY
  12. |     key_len        2
  13. |     ref            sakila.film_actor.actor_id
  14. |     rows           1
  15. +- JOIN
  16.    +- Bookmark lookup
  17.    |  +- TABLE
  18.    |  |  TABLE          film_actor
  19.    |  |  possible_keys  PRIMARY,idx_fk_film_id
  20.    |  +- INDEX lookup
  21.    |     KEY            film_actor->idx_fk_film_id
  22.    |     possible_keys  PRIMARY,idx_fk_film_id
  23.    |     key_len        2
  24.    |     ref            sakila.film.film_id
  25.    |     rows           2
  26.    +- TABLE scan
  27.       rows           1022
  28.       +- TABLE
  29.          TABLE          film
  30.          possible_keys  PRIMARY
  31. 3 rows IN SET (0.00 sec)

Now, that's handy.

What are your favorite ideas?

Posted by Baron Schwartz @ 7:57 pm :: ideas, tips, tools  
Print This Post Print This Post book mark Neat tricks for the MySQL command-line pager in del.icio.usdel.icio.us :: submit Neat tricks for the MySQL command-line pager to digg.comdigg  
Comment RSS  
Related posts: :Logging your MySQL command line client sessions::MySQL 5.1 Command line client improvements::MySQL Query Cache WhiteSpace and comments:
 

8 Comments »

  1. 1. Giuseppe Maxia

    Cool tricks. My most frequent usage of “pager” is “vim -” and then manipulate the output at will.

    I have a handy script that gives me the size of indexes in a database from a SHOW TABLE STATUS. (Useful when you need to calculate how much memory to allocate for a buffer).

    $ cat index_size.sh
    perl -lane ‘$i+= $F[17];END{print $i}’

    mysql> pager ./index_size.sh
    PAGER set to ‘./index_size.sh’
    mysql> show table status;
    71680

    mysql> nopager

    Giuseppe

    Comment :: June 23, 2008 @ 10:38 pm

  2. 2. Parvesh Garg

    This is cool. I have known (and ignored) this earlier, my bad. But something with mk-visual-explain (one of my favorite tools) is just great.

    Next, I would like to see some regex based piping that I can specify in my.cnf, something like

    pager ^explain = mk-visual-explain
    pager ^show engine = less
    pager .* = stdout

    I dont think I’m asking too much :)

    –
    Parvesh

    Comment :: June 24, 2008 @ 2:42 am

  3. 3. PaulM

    This is one I use a bit, requires a Unix/Linux OS though. You could do similar on MySQL on Windows.

    pager grep -v “Sleep” |sort -n -t “|” -k 7 |tail

    show processlist will now suppress sleeping sessions and display the top 10 oldest sessions.

    Thanks for the tip on passing any script… very nice. I have interesting tests to try out… or just use mytop eh?

    Have Fun

    Paul

    Comment :: June 24, 2008 @ 4:32 am

  4. 4. Clint Byrum

    Parvesh, you can have your cake and eat it too.

    superpager.bash:

    #!/bin/bash
    teefile=~/.mysqltee

    goback=100

    # find the last ^mysql>

    lastcmd=$(tail -n $goback $teefile | grep “^mysql> “|tail -1| awk -F”mysql> ” ‘{print $2}’|sed -e ’s/\*/\\*/g’)

    echo lastcmd=$lastcmd

    if echo “$lastcmd” | egrep -qi “^\s*explain” ; then
    exec mk-visual-explain
    fi
    if echo “$lastcmd” | egrep -qi “^\s*show\s+engine” ; then
    fgrep -C3 -i “LOCK” | less
    exit
    fi
    less

    One has to do a \T ~/.mysqltee before the \P superpager.bash , but thats nice because everything you do gets logged..

    Works reasonably well for something I hacked together with bash in 10 minutes. Would be nice if the pager could be passed args like “last command” but this is a decent enough hack until they implement that in version 6.1 of mysql. ;)

    Comment :: June 24, 2008 @ 4:39 pm

  5. 5. Ryan Lowe

    I use pager for two things:

    1) Save the results of the last query
    2) Replace EXPLAIN with mk-visual-explain

    Unfortunately, MySQL 5.0.51a and earlier writes to the tee file AFTER the pager has completed (and to .mysql_history only after you exit mysql), so getting the last command is impossible via these methods (however, it works if you are willing to issue queries twice …). You *can* still override commands (EXPLAIN, for example) by having scripts that error if the input does not conform to a standard; an example follows:

    #!/usr/bin/perl

    use strict;
    use warnings FATAL => ‘all’;
    use English qw ( -no_match_vars );

    my $out;

    while ( my $line = ) {

    $out .= $line;

    }

    # Log results to a file (always useful)
    # But only ever save one :)
    open (PAGER_LOG, ‘>/tmp/mysql_pager.log’);
    print PAGER_LOG “$out\n”;
    close (PAGER_LOG);

    # See if it an EXPLAIN
    eval { local $SIG{’__WARN__’}; system(’/usr/local/bin/mk-visual-explain /tmp/mysql_pager.log 2> /dev/null’); };

    if ($?) {

    print $out;

    }

    Comment :: June 24, 2008 @ 9:34 pm

  6. 6. Ryan Lowe

    gah. The scrubber changed:

    while ( my $line = <STDIN>)

    to

    while ( my $line = )

    Comment :: June 24, 2008 @ 9:35 pm

  7. 7. Javier Tacón

    pager less -niSFX

    Comment :: June 25, 2008 @ 2:59 am

  8. 8. Parvesh Garg

    Check my post http://optimmysql.blogspot.com/2008/07/mysql-command-line-pager-mysmartpager.html for a small hack that can do regex based paging.

    Comment :: July 8, 2008 @ 9:55 am

 

Subscribe without commenting


This page was found by: mysql pager mysql command line mysqlcommand mysql command line c... mysql commandline