Expertiza only support UTF8 characters

[yangsong8]

In current Expertiza db, it only supports utf8 characters. If a user wants to save any non-utf8 characters, either with importing or with typing and saving, will cause system failure.

One thing we can do is to filter out the non-utf8 chars before saving, but the problems, there could be many models/fields got involved.

Another idea is, we can try to make a change on the char set of the DB. This will increase the db size.

[yangsong8]

We are going to migrate the db to postgresql during the summer. Will revisit this issue after that.

[ferryxo]

maybe check if the review text is in UTF-8 before it's submitted

[efg]

Yang says this is a problem for Chinese, who may accidentally type Chinese commas, and then their review is lost.

[johnbumgardner]

Still present

[johnbumgardner]

Here is a UTF 8 error
Despite E1846 being merged this still doesnt work