Percona Server with TokuDB
I was excited to see that TokuDB for MySQL from Tokutek is now open source. I believe this is a great technology and it was a significant restriction on the adoption when it was distributed only in a binary format with a proprietary license.
So I went and built Percona Server 5.5.30 with TokuDB’s storage engine. This is an experimental build and I am just going to play with it. The build procedure is somewhat complicated and confusing and this stage, so I want to share it with the entire MySQL community in case you want to try it out for yourselves.
It’s available from our TESTING download area. The source code used to build this binary is in the same directory.
I have not found at a first glance an answer on obvious question, can it be replacement for InnoDB based application? What redevelopment does it require?
Thanks for putting this together Vadim. I tried to use the released binary last night and could not get INSTALL PLUGIN to work – initialization fails with errors. Hopefully this will be more fruitful.
Otto, please email [email protected] with details on your issue. We have set up that google groups list to deal with such issues.
This is super exciting. We would love to be able to use percona + tokudb with the same binaries.
Thanks Zardosht – I actually just figured out it didn’t like that RHEL 6 having /sys/kernel/mm/redhat_transparent_hugepage/enabled. After issuing echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled from the CLI, I was able to get the plugin to initialize. That isn’t part of the Quick Start Guide, so anyone else using RHEL 6 (and presumably CentOS 6) may have to follow suit.
Thanks Paul. I am glad you were not able to run with huge pages enabled, as that tanks our performance, because it prevents us from accurately tracking our memory usage. I have passed along the feedback that you were not immediately able to diagnose the issue.
You are right about users running RHEL 6 and CentOS 6 will have to follow suit, with our build and probably this Percona build as well.
Sorry for my ignorance, I have the tarball, I decompress, so now?
I’m running on CentOS 5.7 and honestly, I used to install MySQL with rpm packages.
Any help will be appreciated.
Thanks
I follow http://dev.mysql.com/doc/refman/5.5/en/binary-installation.html and seems to be fine, but when I try to run mysql command-line I found this:
# bin/mysql
bin/mysql: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory
What I miss?
I found this:
# ldd mysql
linux-vdso.so.1 => (0x00007fff0f1c0000)
librt.so.1 => /lib64/librt.so.1 (0x00000035c5600000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003c72200000)
libdl.so.2 => /lib64/libdl.so.2 (0x000000309de00000)
libtinfo.so.5 => not found
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00000030b1800000)
libm.so.6 => /lib64/libm.so.6 (0x000000309e600000)
libc.so.6 => /lib64/libc.so.6 (0x000000309da00000)
/lib64/ld-linux-x86-64.so.2 (0x000000309d600000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00000030af800000)
Maybe CentOS it’s a little old for Percona.
yum install ncurses-libs might be the fix for the libtinfo problem.
http://rpmfind.net/linux/rpm2html/search.php?query=libtinfo.so.5
I can start the server but failed to connect the server.
error message:
$ mysql -S /u01/tokudb/run/mysql.sock -uroot
ERROR 2012 (HY000): Error in server handshake
Is there any solution?
Finally I copied this file from a server with a newer CentOS version, and it works.
Crazy.
Hi,
me again. Will Percona Xtrabackup supports tokuDB in the future?
Same error as zhai here…
Not much in the logs… the version strings look a bit broken though
130425 15:07:00 Percona XtraDB (https://www.percona.com) 5.5.30-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-29.3 started; log sequence number 1597971
Version: ‘5.5.30-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-tokudb-7.0.1-tokudp�
’ socket: ‘/var/lib/mysql/mysql.sock’ port: 3306 MySQL Community Server (GPL)
To all, sorry for a delay with my response, I was wrapping up with all things on our conference.
Great too see so much interest and comments, I will answer to some.
1. I re-released the binaries with the fixed version strings. The link is the same.
2. On a weird error at connection. I have no idea what it is yet, make sure you have CentOS 6.x, this is where I compiled. I guess different OS may cause issues.
3. on XtraBackup support… we may have a very basic backup MyISAM-like for TokuDB. Fully functional backup will be available from Tokutek.
FWIW, I tried the Community Edition from Tokutek and that worked great.
http://www.tokutek.com/resources/support/gadownloads/
Tested on Fedora 16 and Fedora 17.
For Fedora users, Fedora 18 binaries are there
https://www.percona.com/redir/downloads/TESTING/Percona-TokuDB/percona-5.5.30-tokudb-7.0.1-fedora-x86_64.tar.gz
I get the following error whenever I try to convert 240 GB InnoDB table to TokuDB:
“ERROR 1030 (HY000): Got error -1 from storage engine”
The table has approximately 1.6 billion rows and I used “ALTER TABLE ENGINE=TokuDB” to do the conversion. Any idea on the possible cause? Both of my two attempts so far ended with the said error. Prior to that, I was successful in converting a 10 GB table with a similar schema.
Building from either Percona’s patch above, or TokuDB for MySQL or TokuDB for MariaDB, I consistently get failure in ./configure:
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
TOKUFRACTALTREE_LIB
linked by target “tokudb” in directory /data/tmp/mariadb-5.5.30-tokudb-7.0.1/storage/tokudb
TOKUPORTABILITY_LIB
linked by target “tokudb” in directory /data/tmp/mariadb-5.5.30-tokudb-7.0.1/storage/tokudb
Any ideas anyone?
Vadim,
Is your build requiring the use of tokudb.build.bash script?
This script consistently fails installation for me in normal tokudb distribution, and I don’t see that it should work on your repackaging. Is it still “./configure && make && make install” on your side?
PJ,
try creating an empty table, then insert data, then create index.
Maybe is better.
Shlomi,
The build procedure is complicated.
I had to modify *make.mysql.bash” script from ft-engine.
Some build instructions you can see in README from: https://github.com/Tokutek/ft-engine
You can find my modification there:
https://gist.github.com/vadimtk/5481807
If nothing helps, I will try to post detailed steps.
@Vadim Tkachenko,
thank you. Am working this out with Tokutek, and documenting all changes.
Shlomi
I’m excited. I’d like to run this on one of my slaves. What do I need to do?
I un-tared and ran mysqd. Found I needed to copy the share folder to my data directory. Double check the ownership of the share directory. I don’t see the tokudb engine.
Mark,
There is no many steps if you are on the good platform (CentOS or Fedora).
I do
scripts/mysql_install_db –datadir=/data/mysql
bin/mysqld –datadir=/data/mysql
and it is all good.
connect from mysql client and check SHOW ENGINES;
Hello,
Leif and I simplified the tokudb build process to a single cmake of the mysql/maria/percona server. These changes are available now in github and will be used in the next point release.
Hi,
Do you have plans for xtrabackup to allow backups from TokuDB the same way it does for XtraBD?
Regards
As an update, see http://www.mysqlperformanceblog.com/2014/03/03/percona-server-5-6-16-64-0-with-tokudb-engine-now-available/ for the new Percona Server 5.6.16-64.0 with TokuDB engine – which is now available as an Alpha release!