Comments on: Converting Character Sets

By: nadavkav

nadavkav — Fri, 03 Apr 2009 17:32:54 +0000

Thank you ! for this valuable post.
But it was a little to late for me, after trying one of those suggestions on the internet
similar to the #2 method category for converting text to binary and back to text again.
We use a Mahara (ePortfolio framework) that was installed with the wrong charset
and got filled with data very quickly, before i had that chance to notice this issue.
eventually (after #2 method did not work properly) i had to manually change the relevant
DB fields
A good ending to this story is that everything works fine, now

By: Shlomi Noach

Shlomi Noach — Wed, 25 Mar 2009 17:26:37 +0000

@Gil,

good question. Will try to set up the configuration and test.

By: Gil

Gil — Wed, 25 Mar 2009 16:53:55 +0000

Thanks @Schlomi

My question was more about what happens if you have latin1 on the master and utf8 on the slave. Same problem?

By: Shlomi Noach

Shlomi Noach — Wed, 25 Mar 2009 16:51:18 +0000

@Gil
If you have a utf8 column on a master, and a latin1 column on a slave, and try to replicate, you may get corrupted data if you insert true utf8 text into the master. The slave will not be able to store utf8 characters and will truncate them.

By: Gil

Gil — Wed, 25 Mar 2009 16:08:38 +0000

In a multi-master replication setup, if I broke replication and ran this script on the passive master, then started replication from active->passive, would the queries fail or do other unwanted things? Are the binary logs charset-sensitive during replication?

By: Shlomi Noach

Shlomi Noach — Mon, 23 Mar 2009 15:36:35 +0000

@Ryan,

Cheers. There’s room for more than one utility per task!

Shlomi

By: Ryan Lowe

Ryan Lowe — Mon, 23 Mar 2009 14:30:34 +0000

@shlomi Had I seen that script sooner, I may have used it! convert_charset has –columns parameter, so it is possible to do one-and-and-only-one column at a time.

@jakub & @pavel This script assumes latin1 data is stored in latin1 columns:)

By: Pavel

Pavel — Mon, 23 Mar 2009 10:54:54 +0000

What happens if a column has mixed latin1 and utf8 at the same time (left from MySQL 4)?
Does MySQL detect that right?

By: Marki

Marki — Thu, 19 Mar 2009 09:54:04 +0000

@Jakub
I’ve just migrated around 300 databases from 4.0 to 5.0. It was not easy, because every database could use different charset…
1) Know the charset of database
2) Export database from mysql 4.0 server
3) Create new empty database on mysql 5.0 with correct default charset
4) Import data into mysql 5.0 server, but prepend data with SET NAMES:
(echo “SET NAMES $chset;”; cat $path/$db.sql) | mysql newdb -hnewserver

By: Shlomi Noach

Shlomi Noach — Thu, 19 Mar 2009 04:18:20 +0000

@Mrten

At any case other examples can be email addresses, local file names (hopefully ASCII), mount points, airports abbreviations, etc. There are quite a few ASCII-only texts, which was my point.

Regards