Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7.9.3 charset problem #3903

Closed
AdminPL opened this issue Jul 18, 2017 · 20 comments

Comments

@AdminPL
Copy link

commented Jul 18, 2017

Issue

After upgradeing suitecrm to version 7.9.3 with polish language pack all special characters ąśźć, etc. are changed to "?" and stored in database this way.

Expected Behavior

Special characters shouldn't change to "?"

Only error i get:
Tue Jul 18 13:55:18 2017 [19165][1][FATAL] Exception handling in /var/www/crm.fortecait.pl/include/MVC/Controller/SugarController.php:397
Tue Jul 18 13:55:18 2017 [19165][1][FATAL] Exception in Controller: Akcja o takiej nazwie nie istnieje.
Tue Jul 18 13:55:18 2017 [19165][1][FATAL] backtrace:
#0 /var/www/crm.fortecait.pl/include/MVC/Controller/SugarController.php(548): sugar_die('Akcja o takiej ...')
#1 /var/www/crm.fortecait.pl/include/MVC/Controller/SugarController.php(1083): SugarController->no_action()
#2 /var/www/crm.fortecait.pl/include/MVC/Controller/SugarController.php(460): SugarController->handleActionMaps()
#3 /var/www/crm.fortecait.pl/include/MVC/Controller/SugarController.php(368): SugarController->process()
#4 /var/www/crm.fortecait.pl/include/MVC/SugarApplication.php(105): SugarController->execute()
#5 /var/www/crm.fortecait.pl/index.php(52): SugarApplication->execute()
#6 {main}

Your Environment

  • SuiteCRM Version used: 7.9.3
  • Browser name and version Chrome Version 51.0.2704.63 (64-bit)
  • Operating System and version Win 10 Home all latest update installed
@horus68

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

Please confirm: this only apply to words in fields, not to the actual translated interface of SuiteCRM, right?

Also reported here for Korean language
https://suitecrm.com/forum/suitecrm-7-0-discussion/15088-korean-language-destroyed-when-save-on-any-text-relative-fields-after-upgraded-to-7-9-3

@AdminPL

This comment has been minimized.

Copy link
Author

commented Jul 18, 2017

Yes, only word fields.

@pgorod

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

For reference, those error messages in English are these

include/MVC/Controller/SugarController.php:548: 
    sugar_die($GLOBALS['app_strings']['LBL_NO_ACTION']);

include/language/en_us.lang.php:1088: 
   'LBL_NO_ACTION' => 'There is no action by that name.',

@AdminPL when you say these values "are stored in the database this way", did you really go into MySQL and check? Or do you just mean when you retrieve them from SuiteCRM app they show wrong?

@AdminPL

This comment has been minimized.

Copy link
Author

commented Jul 18, 2017

I checked in database, thought it may be comparing charset problem

@pgorod

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

And did you upgrade PHP at the same time? Or MySQL? Or other Linux packages?

@AdminPL

This comment has been minimized.

Copy link
Author

commented Jul 18, 2017

No other upgrade's same time, we turned off all auto upgrade after 3 weeks ago php went to latest version an we needed to install version 7 again.

@AdminPL

This comment has been minimized.

Copy link
Author

commented Jul 18, 2017

Temporary i fixed issue by replacing include folder from version 7.9.2 on testing installation

@pgorod

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

Interesting, that fix... can you narrow it down to just a single file?

Compare the two directories and replace only a single file that fixes the problem?

@AdminPL

This comment has been minimized.

Copy link
Author

commented Jul 18, 2017

I tried to switch only files which i found in error log but no effect, so at best i can compare and get back to you with list which files did changed after upgrade, but not sure if it will truly resolve the problem.

Also what i did may affect another functions so i don't recomend it to anyone unless absolutely necessary.

@pgorod

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

@horus68 in the upgrade package from 7.9.x to 7.9.3 I see that string defined like this:

'LBL_NO_ACTION' => 'There is no action by that name: %s',

This is the commit merged in 7.9.1 that touches this part of the code. It was something @gymad was working on.

@horus68

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

@pgorod crowdin languages are for version 7.9.2
It will take some hours to update to 7.9.3 as this is a massive language update (version 7.9.3 includes some big PR for copyright and unused language strings) but I also need to compare for changes on my local "advanced" language master branch to not remove crowdin strings from PR not merged yet.

@pgorod

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

@horus68 I just corrected my post above, I was mistaken. The string is updated on my 7.9.3. Sorry.

Maybe I'm also wrong about this being present since 7.9.1. Anyway, once you have time to update the translations on Crowdin, that %s will have to be present for any system that has the new code in gymad's commit, otherwise the error will come up.

We still have to determine whether the error in the logs is indeed related to the charset bug described. They might be different things.

AdminPL can you please try reverting to the include directory contents that were giving the error, and instead change your Polish language file that contains LBL_NO_ACTION so that it has a %s in it?

'LBL_NO_ACTION' => 'Akcja o takiej nazwie nie istnieje: %s',

@AdminPL

This comment has been minimized.

Copy link
Author

commented Jul 18, 2017

Can't do it right now, VM with test system is down, i'll do it ASAP, propably tommorow.

@horus68

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

I found that the actual string on Crowdin already includes the % in that string. So if someone uses a language pack version 7.9.2.0 and up) will not have that issue.

@pgorod

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

Ok everyone I have some not very good news about this...

  • this has nothing to do with language packs, it can be reproduced on the live demo and on my system without any language pack
  • the error we've been discussing above is misleading - forget about it, if it's a problem, it's a different problem. This problem does not leave traces in the logs (at least I couldn't find any).

The simplest steps to reproduce:

  1. click create Contact
  2. fill the Last name field (which is the only mandatory field) with some values
  3. hit Save
  4. the values are not correctly stored in the database, and so show incorrectly also on screen

The values that produce problems are basically anything that is not the standard english character:

  • I tried the name Gonçalves, it turned to Gonlves. Other portuguese characters like ã get the same treatment, the character disappears, and takes the following character along with it...
  • I tried the polish ąśźć listed above, it turns to ????

So basically 7.9.3 somehow broke SuiteCRM's Unicode capabilities in every field of every screen...

@samus-aran

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2017

Hi Everyone.

We'll raise this as a high priority and deal with this and produce a patch immediately. I believe the issue has came about dealing with a Security issue, But when we know more we'll update the issue and provide a resolve.

Be in touch soon.

@gymad

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2017

Hi @pgorod, I have a commit with a possible fix. Can you please confirm that it solves the issue?

@pgorod

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2017

@gymad I have a meeting starting now, but in a five minute test, it did seem to solve everything. Those two tests above (with the portuguese and the polish characters) seem to be ok with that code change 👍
Text was showing correctly in phpMyAdmin and on screen.

@AdminPL

This comment has been minimized.

Copy link
Author

commented Jul 19, 2017

Sam situation on my instance. Checked also with german character's seems to be working fine.

samus-aran added a commit that referenced this issue Jul 19, 2017
Merge pull request #3912 from gymad/fix_3903_charset
Fixed #3903 - use correct charset
@halest

This comment has been minimized.

Copy link

commented Jul 19, 2017

Seems to be working

mattlorimer added a commit that referenced this issue Sep 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.