July 31, 2014

Solving RPM installation conflicts in CentOS 5 and CentOS 6

Lately we’ve had many reports of the RPM packages for CentOS 5 (mostly) and CentOS 6 having issues when installing different combinations of our products, particularly with Percona Toolkit for MySQL. Examples of bugs related to these issues are lp:1031427 and lp:1051874.

These problems arise when trying to install a package from the distribution that is linked against the version of libmysqlclient.so shipped by the distribution (libmysqlclient.so.15 for CentOS 5/libmysqlclient.so.16 for CentOS 6) and a version of Percona Server that depends on another version of libmysqlclient.so, usually more recent. Bug lp:1031427 is an example of this, and shows how the packages would conflict when trying to install libmysqlclient.so.

For example, when installing php-mysql alongside PS 5.5 in CentOS 6:


# yum -q install Percona-Server-server-55 php-mysql

Installing:
Percona-Server-server-55 x86_64 5.5.29-rel29.4.401.rhel6 percona 15 M
php-mysql x86_64 5.3.3-14.el6_3 updates 79 k
Installing for dependencies:
Percona-Server-client-55 x86_64 5.5.29-rel29.4.401.rhel6 percona 7.0 M
Percona-Server-shared-51 x86_64 5.1.67-rel14.3.506.rhel6 percona 2.8 M
Percona-Server-shared-55 x86_64 5.5.29-rel29.4.401.rhel6 percona 787 k

Transaction Summary
=====================================================================================================================================================
Install 5 Package(s)

Is this ok [y/N]: y

Transaction Check Error:
file /usr/lib64/libmysqlclient.so conflicts between attempted installs of Percona-Server-shared-51-5.1.67-rel14.3.506.rhel6.x86_64 and Percona-Server-shared-55-5.5.29-rel29.4.401.rhel6.x86_64
file /usr/lib64/libmysqlclient_r.so conflicts between attempted installs of Percona-Server-shared-51-5.1.67-rel14.3.506.rhel6.x86_64 and Percona-Server-shared-55-5.5.29-rel29.4.401.rhel6.x86_64

The traditional solution for this situation was to provide a special package, Percona-Server-shared-compat (modeled after upstream’s MySQL-shared-compat) which would contain ALL versions of libmysqlclient.so.* together and wouldn’t conflict. Probably some of you are familiar with this approach.


# yum -q install Percona-Server-server-55 Percona-Server-shared-compat php-mysql

Installing:
Percona-Server-server-55 x86_64 5.5.29-rel29.4.401.rhel6 percona 15 M
Percona-Server-shared-compat x86_64 5.5.29-rel29.4.401.rhel6 percona 3.4 M
php-mysql x86_64 5.3.3-14.el6_3 updates 79 k
Installing for dependencies:
Percona-Server-client-55 x86_64 5.5.29-rel29.4.401.rhel6 percona 7.0 M
Percona-Server-shared-55 x86_64 5.5.29-rel29.4.401.rhel6 percona 787 k

Transaction Summary
=====================================================================================================================================================
Install 5 Package(s)

Notice how PS-shared-compat installs along the -shared package, providing the older libmysqlclient.so.16 required by php-mysql.

However, this has proved non-intuitive and problematic, since the shared-compat package wouldn’t get selected unless explicitely installed — and many of our users would rather have it “just work” without requiring additional knowledge of what the particular workaround was, etc..

We’re now trying a solution in which our -shared packages won’t conflict anymore at libmysqlclient.so, so we are able to install them side-by-side, modelled after the mysql-libs packages provided by CentOS/Redhat. So even if the user wants to install PS 5.5 alongside packages that depend on 5.1/5.0, the -shared packages will work together. For example installing 5.5 and postfix in CentOS:


# yum -q install Percona-Server-server-55 postfix

Installing:
Percona-Server-server-55 x86_64 5.5.29-rel29.4.402.rhel5 percona-testing 19 M
postfix x86_64 2:2.3.3-6.el5 base 3.8 M
Installing for dependencies:
Percona-SQL-shared-50 x86_64 5.0.92-b23.89.rhel5 percona-testing 1.8 M
Percona-Server-client-55 x86_64 5.5.29-rel29.4.402.rhel5 percona-testing 9.1 M
Percona-Server-shared-55 x86_64 5.5.29-rel29.4.402.rhel5 percona-testing 993 k

 

… and this will install without problems.

Additionally, this has the advantage of allowing an upgrade from 5.1 to 5.5 without uninstalling any software that depended on the old version.


# rpm -qa | grep ^Percona
Percona-Server-client-51-5.1.67-rel14.3.507.rhel6.x86_64
Percona-Server-shared-51-5.1.67-rel14.3.507.rhel6.x86_64
Percona-Server-server-51-5.1.67-rel14.3.507.rhel6.x86_64

In this case only Percona-Server-client-51 and Percona-Server-server-51 need be removed, allowing any package that depends on Percona-Server-shared-51 (providing libmysqlclient.so.16) to remain installed. After the server and client packages are uninstalled, you can install PS 5.5 without conflict.

The current package candidates for versions 5.0.92 (which required an update), 5.1.67-14.3 and 5.5.29-29.4 can be tested from the percona-testing repository. We encourage you to try these out and send us your feedback and/or file any bugs you find.

Installation instructions for Percona Testing repositories.

We’re aiming to include these fixes in our next releases of Percona Toolkit 5.1 and Percona Toolkit 5.5. Percona Toolkit users in particular will enjoy this update since it’ll mean no more trouble when installing it from repository!

Comments

  1. You rock my repos Ignacio! Thanks so much! :)

  2. guly says:

    this is not limited to centos, i had some issues also on debian/ubuntu (don’t know if latest packages works better for dependencies, sorry). while you’re fixing rpm spec file, could you please have a look also to deb ? thanks :)

  3. Ignacio Nin says:

    Hey guly!

    Our debian packages already use these approach — based in the original distribution files for 5.1 in which we based our percona packages.

    Of course, this doesn’t mean they’re free of issues :) Have you experienced any when installing our software? If so, please let me know of the particular problem and I’ll take a look into it.

    Thanks!

  4. hailong guo says:

    hello,
    nice to meet you!
    now,i meet a trouble and need your help!
    i setup percona cluster in vm, if in file /etc/my.cnf,i add wsrep_cluster_address=gcomm:// to fill it ,the mysql server will not running

    [root@mysql1 ~]# vi /etc/my.cnf
    [mysqld]
    wsrep_provider=/usr/lib/libgalera_smm.so
    user=mysql
    wsrep_cluster_address=gcomm://
    binlog_format=ROW
    wsrep_slave_threads=2
    wsrep_cluster_name=trimethylxanthine
    wsrep_sst_method=rsync
    wsrep_node_name=mysql1
    innodb_locks_unsafe_for_binlog=1
    innodb_autoinc_lock_mode=2

    the wrong infomaton :

    130221 17:59:00 mysqld_safe WSREP: Running position recovery with –log_error=/tmp/tmp.cuCEbd7883
    130221 17:59:05 mysqld_safe WSREP: Recovered position 00000000-0000-0000-0000-000000000000:-1
    130221 17:59:05 [Note] WSREP: wsrep_start_position var submitted: ’00000000-0000-0000-0000-000000000000:-1′
    130221 17:59:05 [Note] WSREP: Read nil XID from storage engines, skipping position init
    130221 17:59:05 [Note] WSREP: wsrep_load(): loading provider library ‘/usr/lib/libgalera_smm.so’
    130221 17:59:05 [Note] WSREP: wsrep_load(): Galera 2.3(r143) by Codership Oy loaded succesfully.
    130221 17:59:05 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
    130221 17:59:05 [Note] WSREP: Reusing existing ‘/var/lib/mysql//galera.cache’.
    130221 17:59:05 [Note] WSREP: Passing config to GCS: base_host = 192.168.14.150; base_port = 4567; cert.log_conflicts = no; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 16; gcs.fc_master_slave = NO; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 2147483647; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = NO; replicator.causal_read_timeout = PT30S; replicator.commit_order = 3
    130221 17:59:06 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
    130221 17:59:06 [Note] WSREP: wsrep_sst_grab()
    130221 17:59:06 [Note] WSREP: Start replication
    130221 17:59:06 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
    130221 17:59:06 [Note] WSREP: protonet asio version 0
    130221 17:59:06 [Note] WSREP: backend: asio
    terminate called after throwing an instance of ‘gu::NotFound’
    01:59:06 UTC – mysqld got signal 6 ;
    This could be because you hit a bug. It is also possible that this binary
    or one of the libraries it was linked against is corrupt, improperly built,
    or misconfigured. This error can also be caused by malfunctioning hardware.
    We will try our best to scrape up some info that will hopefully help
    diagnose the problem, but since we have already crashed,
    something is definitely wrong and this may fail.
    Please help us make Percona Server better by reporting any
    bugs at http://bugs.percona.com/

    key_buffer_size=0
    read_buffer_size=131072
    max_used_connections=0
    max_threads=151
    thread_count=0
    connection_count=0
    It is possible that mysqld could use up to
    key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 329809 K bytes of memory
    Hope that’s ok; if not, decrease some variables in the equation.

    Thread pointer: 0×0
    Attempting backtrace. You can use the following information to find out
    where mysqld died. If you see no messages after this, something went
    terribly wrong…
    stack_bottom = 0 thread_stack 0×30000
    /usr/sbin/mysqld(my_print_stacktrace+0×33)[0x8415893]
    /usr/sbin/mysqld(handle_fatal_signal+0x48c)[0x82dee3c]
    [0x8e8420]
    /lib/libc.so.6(abort+0×101)[0xb1a701]
    /usr/lib/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0×150)[0x710bab0]
    /usr/lib/libstdc++.so.6[0x7109515]
    /usr/lib/libstdc++.so.6[0x7109552]
    /usr/lib/libstdc++.so.6[0x710968a]
    /usr/lib/libgalera_smm.so(_ZNK2gu3URI10get_optionERKSs+0xf6)[0xe180b6]
    /usr/lib/libgalera_smm.so(_ZN5gcomm5paramIN2gu8datetime6PeriodEEET_RNS1_6ConfigERKNS1_3URIERKSsSB_PFRSt8ios_baseSD_E+0xfc)[0xe80e9c]
    /usr/lib/libgalera_smm.so(_ZN5gcomm2PCC2ERNS_8ProtonetERKN2gu3URIE+0xc2)[0xea03d2]
    /usr/lib/libgalera_smm.so(_ZN5gcomm9Transport6createERNS_8ProtonetERKN2gu3URIE+0x2be)[0xebbece]
    /usr/lib/libgalera_smm.so(_ZN9GCommConn7connectERKSs+0xeb)[0xf07d8b]
    /usr/lib/libgalera_smm.so[0xf04970]
    /usr/lib/libgalera_smm.so(gcs_core_open+0×88)[0xefa248]
    /usr/lib/libgalera_smm.so(gcs_open+0x2c8)[0xf01028]
    /usr/lib/libgalera_smm.so(_ZN6galera13ReplicatorSMM7connectERKSsS2_S2_+0x2b4)[0xf49904]
    /usr/lib/libgalera_smm.so(galera_connect+0xae)[0xf6675e]
    /usr/sbin/mysqld(_Z23wsrep_start_replicationv+0×107)[0x8294897]
    /usr/sbin/mysqld(_Z18wsrep_init_startupb+0x7c)[0x829551c]
    /usr/sbin/mysqld[0x813d7ec]
    /usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x81a)[0x8140c0a]
    /usr/sbin/mysqld(main+0×27)[0x8132e67]
    /lib/libc.so.6(__libc_start_main+0xdc)[0xb05e9c]
    /usr/sbin/mysqld[0x8132d81]
    You may download the Percona Server operations manual by visiting
    http://www.percona.com/software/percona-server/. You may find information
    in the manual which will help you identify the cause of the crash.
    130221 17:59:06 mysqld_safe mysqld from pid file /var/lib/mysql/mysql1.pid ended

    can you help me to solve this program?

  5. And with the mysql vs. /usr/bin/mysql changes in the Oracle packages it will even get harder to get the correct dependencies:
    https://bugs.launchpad.net/percona-xtrabackup/+bug/1095972

  6. What we (openSUSE) do is that ‘*.so’ file is only in devel package and libraries are in ‘various packages with shared libraries each one providing only ‘*.so.*’ files. Unfortunatelly that doesn’t solve problems if you have multiple libraries with same version like having distribution compiling against Oracles MySQL (and wanting to play it safe) and wanting to use MariaDB. I ended up with renaming the library on non-default databases, just to be sure…

Speak Your Mind

*