Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize language names and codes #2440

Open
wants to merge 5 commits into
base: master
from
Open

Conversation

@trembyle
Copy link
Contributor

trembyle commented Sep 1, 2020

Please see the additional documentation in language.cpp. The goal is to use more consistent naming conventions based wherever possible on ISO standards.

Some existing entries needed to be changed in common and also in individual engines. I was able to build successfully and detect and run games in these languages. I'm looking for feedback on this. Is there anything else that I need to consider?

trembyle added 3 commits Sep 1, 2020
* Add documentation to add or maintain language values
* Fix and reorder existing entries to match ISO standards
* Further change may be needed to fix language constants in engines
* Use ISO-3166 country codes and ISO-639 language codes
* Use ISO-3166 country codes and ISO-639 language codes
@sev-
Copy link
Member

sev- commented Sep 1, 2020

Let's imagine I already added Chinese DIG to ScummVM (I really did yesterday). It is stored in scummvm.ini with 'language=zh', which was ZH_TWN. Now, that language is parsed to ZH_ANY. But in SCUMM engine code there are no checks against ZH_ANY, and the game will stop working.

Thus, a separate step for converting scummvm.ini is required. We did that in the past when we made some targets obsolete. See engines/obsolete.cpp. In order to avoid multiple conversions, I would do the following: Check versioninfo field in the INI file, if it is 2.2.* or less, then iterate over all targets and convert language, then save the file. Saving file will update versioninfo, and you will avoid double conversion.

common/language.h Outdated Show resolved Hide resolved
{ "ru", "ru_RU", "Russian", RU_RUS },
{ "sr", "sr_RS", "Serbian", SR_SRB },

This comment has been minimized.

@lolbot-iichan

lolbot-iichan Sep 1, 2020 Contributor

Maybe we need "sr_RS@latin" here, since "sr_RS" seems to be cyrillic locale and the only currently supported game in Serbian is using latin script.

This comment has been minimized.

@trembyle

trembyle Sep 2, 2020 Author Contributor

The right way might be to include another parameter for codeset. We also don't differentiate between Traditional and Simplified Chinese either, but it looks like both are available in Wintermute.

This list has some more examples:
https://docs.oracle.com/cd/E23824_01/html/E26033/glset.html

This comment has been minimized.

@lolbot-iichan

lolbot-iichan Sep 2, 2020 Contributor

We also don't differentiate between Traditional and Simplified Chinese either

In WME, "Chinese (China)" and "Chinese (Taiwan)" are used in "Simplified Chinese" and "Traditional Chinese" meaning respectfully. As far as I understand, those things are pretty much the same:

  • Simplified Characters were introduced by China and are also used at Malaysia and Singapore, China is much bigger then Malaysia and Singapore in both area and population.
  • Traditional Characters are still used at Taiwan, Hong Kong and Macau, Taiwan is much bigger then Hong Kong and Macau in both area and population.

I guess "Chinese (China)" and "Chinese (Taiwan)" can be renamed to "Chinese (Simplified)" and "Chinese (Traditional)", if needed.

ZH_ANY was added as a workaround for a WME game Reversion that at some moment contained 3 Chinese translation packages in a single game directory after an update: new Simplified and Traditional translations were added and old Chinese package was not deleted (and detected by detection.cpp). To fix this, we decided to mark all old Reversion Chinese games as ZH_ANY ("generic" Chinese) and use ZH_CHA/ZH_TWN for Simplified/Traditional.

@lolbot-iichan
Copy link
Contributor

lolbot-iichan commented Sep 2, 2020

Thus, a separate step for converting scummvm.ini is required. We did that in the past when we made some targets obsolete. See engines/obsolete.cpp. In order to avoid multiple conversions, I would do the following: Check versioninfo field in the INI file, if it is 2.2.* or less, then iterate over all targets and convert language, then save the file. Saving file will update versioninfo, and you will avoid double conversion.

Besides ConfMan's language records and AdvancedDetector target names, there is one more scenario: engines/glk/quetzal.cpp is writing language codes to savenames. Not sure if it's fine or not.

@trembyle
Copy link
Contributor Author

trembyle commented Sep 2, 2020

Besides ConfMan's language records and AdvancedDetector target names, there is one more scenario: engines/glk/quetzal.cpp is writing language codes to savenames. Not sure if it's fine or not.

Since this is the first public release to include GLK, users shouldn't expect saved games from prior development builds to work in stable 2.2.0.

@trembyle trembyle force-pushed the trembyle:language branch from 6f80473 to 92aae82 Sep 2, 2020
@trembyle
Copy link
Contributor Author

trembyle commented Sep 2, 2020

Thus, a separate step for converting scummvm.ini is required. We did that in the past when we made some targets obsolete. See engines/obsolete.cpp. In order to avoid multiple conversions, I would do the following: Check versioninfo field in the INI file, if it is 2.2.* or less, then iterate over all targets and convert language, then save the file. Saving file will update versioninfo, and you will avoid double conversion.

I think I got close here, but it's not quite working yet. I'm unable to detect some games with new codes. Can you see what I'm doing wrong?

@sev-
Copy link
Member

sev- commented Sep 2, 2020

What are "some" games?

@sev-
Copy link
Member

sev- commented Sep 2, 2020

Besides ConfMan's language records and AdvancedDetector target names, there is one more scenario: engines/glk/quetzal.cpp is writing language codes to savenames. Not sure if it's fine or not.

Since this is the first public release to include GLK, users shouldn't expect saved games from prior development builds to work in stable 2.2.0.

No, this change cannot go to 2.2.0, it is way too risky so close before the release.

@trembyle
Copy link
Contributor Author

trembyle commented Sep 2, 2020

Besides ConfMan's language records and AdvancedDetector target names, there is one more scenario: engines/glk/quetzal.cpp is writing language codes to savenames. Not sure if it's fine or not.

GLK has only one game in an affected language (nissen). This is only available in Danish, and the two character code for that language is not changing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.