Change MySQL UTF-8 examples to use utf8mb4

[DHager]

You might think MySQL's utf8 is the right choice, but it's actually got some problems handling certain character inputs. The later, corrected mode of utf8mb4 has fewer surprises.

[xabbuh]

You will have to indented all these lines by four spaces to be rendered inside the enclosing sidebar block.

[DHager]

Hmm, I guess I did want to have a caution-block inside the sidebar-block, but offhand I'm not sure if that's a thing that is done elsewhere in the documentation.

[xabbuh]

@DHager Thanks for your suggestion. Do you have some resources to which we can refer here?

[DHager]

@xabbuh Sure. First, MySQL 5.5 docs on on the 10.1.10.6 The utf8mb4 Character Set:

The character set named utf8 uses a maximum of three bytes per character and contains only BMP characters. As of MySQL 5.5.3, the utf8mb4 character set uses a maximum of four bytes per character supports supplemental characters [...]

For a supplementary character, utf8 cannot store the character at all

In addition, other people have reported that the failure mode is complete string truncation, losing everything past the first problem-symbol:

The content got truncated at the first astral Unicode symbol [...] MySQL returned a warning message

[DHager]

Crap, an unrelated doc-fix seems to have become part of the pull-commit, I forgot Github automatically drew them in. I'll see if I can fix it.

[xabbuh]

In restructeredText, you have to use double backticks.

[DHager]

OK, my local make html still doesn't have the same styles as the main site, but they should look like code/keywords now.

[weaverryan]

Hi @DHager!

Thanks for bringing this up - I think it's a good note, especially since it causes silent issues (truncations).

Since this character set is new to 5.5.3, I think we should:

A) Add 2 new (commented out) lines in the [mysqld] codeblock with the old character set, and a comment above it that says you need to use this before 5.5.3
B) Add a quick comment right below the [mysqld] line (so above where you set the character set to your new type) that explains (shortly) that this character set is better, with maybe a link to the MySQL docs about this
C) Remove the caution block you added - because this will basically be covered by the above comment.

What do you think? If you agree, can you make these changes?

Thanks!

[ricardclau]

I would personally not add the commented lines but reword a bit the caution section adding a link to the official MySQL docs.

MySQL 5.5.3 was launched 5 years ago and I think it is good that this setting becomes more popular as many people (myself included until I read this PR) still believe utf8 is the way to go

Plus, I think we should take the opportunity to review all the Symfony and Silex docs regarding this. A good opportunity to get easy doc badges as well :)

Just my 2 cents @weaverryan and @DHager :)

[DHager]

@weaverryan, @ricardclau : I'm inclined to put utf8mb4 as the default because:

1. If it isn't supported by their DB, the user will find out immediately.
2. The string is very distinctive, so they can easily find the configuration-file and search online for more information.

The reverse-case, using utf8, is trickier:

1. Everything will seem to work until weeks, months, or years later.
2. They won't necessarily know/remember to consult this documentation or their my.cnf comments. Instead they will probably waste time searching for "mysql text truncated" and (if they saw the MySQL warnings) "mysql incorrect string value".

I've "promoted" the caution block to another sentence, and added comments to the configuration sample.

[ricardclau]

I am 100% with @DHager on this one, but of course up to you @weaverryan :)

[weaverryan]

I think it's great now - you're right that the new one should be the default, and I like that you show the old one and have some (short) words explaining. I'll merge this shortly :).

Thanks!

[ricardclau]

@DHager do you want to add this to the Silex / Doctrine docs as well? I can have a look at them but since you opened this PR I think it is fair that you go for them if you have time

[weaverryan]

Thanks again! And yes, I think adding this to Silex or Doctrine if they have similar notes will make great sense. Cheers!

[DHager]

Forgot to mention: Using utf8mb4 does have one risk, which I believe to be minor/easily-detectable. Depending on a lot of MySQL configuration details, it's possible that certain text-columns cannot be indexed/unique because their new byte-requirements are larger than what the engines (e.g. innodb) are configured to support.

I'd call this minor because if someone already has a database, those columns will already be in some other character set, and if they're creating a new one... they probably shouldn't be trying to index longer text fields in the first place.