Installer: Force Database UTF8 Collation

[dakur]

Hi, we are running HumHub on Ubuntu 14.04 and asking for user authentication on LDAP running on Windows 2003 Server. But there is a problem with received information about the user - the lastname, which containts special national characters, is not imported correctly - it shows "kucha?" instead of "kuchař" in HumHub. Is there any possibility to set characters encoding, or something like that?

[jkuchar]

I'm @dakur college.

It looks like something more global than just LDAP issue. When I post a text on a wall there is the same issue.

ěščřžýáíéů becomes ?š??žýáíé?

It looks like wrong character-set in database.

• There definitely should be utf-8, is there? If not why?
• Shoudn't it be default in HumHub?
• Tables should be created with utf8 encoding even if whaever database cofiguration is, am I right?

[dakur]

Then, maybe it is something wrong on our side, because this goes well on other installations. Will check it out.

[luke-]

Hmm, no problem here with special chars and ldap.

Are your tables Profile/User set to UTF8?

[jkuchar]

We will check this thing more deeply with @dakur next week. Probably there is some bug in installation script. Because it should ensure, that tables are created in utf8 (even if database configuration is whatever)

[dakur]

Ok, the problem was, that I did not follow the steps in installation guide about creating database. Instead, I've created user "humhub" and checked the option "create database with the same name as username" in phpMyAdmin. The default collation is "latin1_swedish_ci", so all HumHub tables were set in "latin1_swedish_ci".
BUT should not the HumHub installator check this out and set the right collation?

[jkuchar]

Or shouldn't installer stop installation and throw an error?

[jkuchar]

@dakur Please, could you update the issue title?

[cboulanger]

I just found out about this problem and will need to convert our databases manually... Please address this issue before releasing v1.0!

[buddh4]

@luke- this can be closed?

[jkuchar]

Still think that this should at least throw an error. So user will not end up in broken state.

[buddh4]

Maybe we can add a prerequisite check for this. What do you think @luke- .

[luke-]

@buddh4 Yes, we may can also check table collations/engines in the test.

[luke-]

In the setup we have to do this check again separately, because the prerequisite check runs before.

It would also be good to make the port of the database configurable.
#158

In addition, a checkbox that creates the database directly, if possible, would also be a good thing.

[buddh4]

@luke- I've opened a seperate issue: #4818

[yurabakhtin]

@luke- I have tested this:

• Created a new DB with collection latin1_general_ci
• Run install
• The collation of the DB was converted to utf8mb4_unicode_ci and all tables were created with same utf8mb4_unicode_ci collection

It works like this because this migration 4ef22e1#diff-f5897f5c0c6fbea496d34d10efdda6c776f533755cf12156b69c76b21366ec7cR8

I have also tested an install without the migration, in such case all tables were created with latin1_general_ci, so the sample ěščřžýáíéů is stored as ?š??žýáíé.

[yurabakhtin]

It would also be good to make the port of the database configurable. #158

Commit dd9a7a7:

[yurabakhtin]

In addition, a checkbox that creates the database directly, if possible, would also be a good thing.

Commit 89e675e:

[luke-]

@yurabakhtin Looks good. Can you also check the collation?

[yurabakhtin]

@luke-

Looks good. Can you also check the collation?

Sorry, but I don't understand what collation check you mean here.

If you tell about collation prerequisite check then I implemented this here #4818.

If you want some additional check during install then I tested the install into DB with wrong collection here #677 (comment), i.e. if DB has no collection utf8mb4 then such DB will be converted to utf8mb4 during install and all tables will be created in the same collation because of migration 4ef22e1#diff-f5897f5c0c6fbea496d34d10efdda6c776f533755cf12156b69c76b21366ec7cR8.

If this is not enough please give me more details where the collation check should be implemented, thanks.

[luke-]

Oh, that's right, I hadn't considered the migration. Then that is enough of course. Thanks!