Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow using unicode characters in world names regardless of system language on Windows 10 1903 and newer versions #51475

Merged
merged 1 commit into from
Sep 12, 2021

Conversation

Qrox
Copy link
Contributor

@Qrox Qrox commented Sep 8, 2021

Summary

Bugfixes "Allow using unicode characters in world names regardless of system language on Windows 10 1903 and newer versions"

Purpose of change

image
image

Windows by default uses the code page corresponding to the system language as the active code page. Many code pages use charsets that support limited unicode characters. For example, the simplified Chinese charset GBK is missing the copyright symbol "©". So when you try to ridicule Windows by asking it to create a world with the name in the image, it throws a fit and miserably fails.

Describe the solution

According to this article, as of Windows 10 1903 you can use an application manifest to switch an application to the utf-8 code page. For older Windows versions, it falls back to using the default system-language-specific code page, and the game will still work, but only characters in the active code page can be used for world names.

Testing

Created worlds with names containing English, Chinese, the copyright symbol ©, and the cow emoji 🐂 (U+1F402, outside the basic multilingual plane), all of which succeeded. (The font doesn't render the cow emoji correctly though)

Game still runs on Windows XP, but it fails as before when saving/loading files with paths containing characters not in the active code page, as expected.

Additional context

Maybe we can make it fail more gracefully when creating a file with a character not in the active code page on older Windows versions.

As a bonus, the application manifest now makes the crash popup more consistent with the system UI style. It's always more comforting when you see a prettier crash window, I guess.

@BrettDong BrettDong added [C++] Changes (can be) made in C++. Previously named `Code` OS: Windows Issues related to Windows operating system labels Sep 8, 2021
@BrettDong
Copy link
Member

Does this also solve the problem of not being able to create world with CJK characters on English version Windows 7, or the world name will then be empty?

@Qrox
Copy link
Contributor Author

Qrox commented Sep 9, 2021

Only Windows 10 1903 onwards supports setting the active code page this way. I didn't test it but I think on older versions including Winodws 7 it will behave as before. I.e. the folder is created because the code calls the wide char version of the Windows filesystem functions, but file creation fails in MinGW-compiled versions because the path is mangled after converting from utf-8 to the active code page.

@Qrox Qrox marked this pull request as draft September 10, 2021 06:34
@Qrox Qrox changed the title Allow using any character in world names (in newer Windows versions) regardless of system language Allow using unicode characters in world names (in newer Windows versions) regardless of system language Sep 10, 2021
@Qrox Qrox marked this pull request as ready for review September 10, 2021 16:12
@Qrox Qrox changed the title Allow using unicode characters in world names (in newer Windows versions) regardless of system language Allow using unicode characters in world names regardless of system language on Windows 1903 and newer versions Sep 10, 2021
@Qrox Qrox changed the title Allow using unicode characters in world names regardless of system language on Windows 1903 and newer versions Allow using unicode characters in world names regardless of system language on Windows 10 1903 and newer versions Sep 10, 2021
@Night-Pryanik
Copy link
Contributor

Should this close #28756?

@Qrox
Copy link
Contributor Author

Qrox commented Sep 12, 2021

Hard to say, the OP didn't provide enough detail. Anyway, the ultimate fix would be using the wchar_t * constructor of fstream from c++17, and this is just a temporary fix that works for newer Windows versions.

@kevingranade kevingranade merged commit 21435d2 into CleverRaven:master Sep 12, 2021
@Qrox Qrox deleted the acp branch September 18, 2021 05:07
Venera3 pushed a commit to Venera3/Cataclysm-DDA that referenced this pull request Sep 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[C++] Changes (can be) made in C++. Previously named `Code` OS: Windows Issues related to Windows operating system Translation I18n
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants