New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OSX] Non-ASCII characters render as garbage when default OS language is CJK #22355

Closed
sxmichaels opened this Issue Nov 5, 2017 · 6 comments

Comments

Projects
None yet
4 participants
@sxmichaels
Contributor

sxmichaels commented Nov 5, 2017

Game version: Experimental 2a3206ac63d9ab907f2036bed07328aae3bdbaf5

Operating system:

  • OS X 10.12.6
  • uname -a output: Darwin rmbp111 16.7.0 Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64

Tiles or curses: Curses

Mods active: None

Other info:

  • gettext 0.19.8.1 from Homebrew
  • Cataclysm compiled with make NATIVE=osx OSX_MIN=10.12 LOCALIZE=1 LANGUAGES=all RELEASE=1 CLANG=1.

Expected behavior

Non-ASCII characters such ß (the German double-S) and Chinese characters render as they normally appear.

Actual behavior

These characters render as follows. The example below is from the Chinese version. It appears the right characters sometimes appear — possibly every other character is right, although I'm not sure if it's that consistent. For instance, "Simplified Chinese" is 中文天朝 , and CDDA displays it as 中 (garbage) 天 (garbage).

Other notes

  • This is still the case even in English mode — ß in the randomly-generated name "Tobias Strauß" (or something like that) also renders as garbage.
  • The Cataclysm logo is still in English, whereas I think it should be 大灭灾 as it was a few versions ago; this may be related.
  • If I build with LOCALIZE=0, the language names in the settings screen display properly, but obviously then I can't actually use them.

2017-11-05 11 13 36

2017-11-05 11 13 56

Steps to reproduce the behavior

Build Cataclysm with make NATIVE=osx OSX_MIN=10.12 LOCALIZE=1 LANGUAGES=all RELEASE=1 CLANG=1. Run make LANGUAGES="zh_CN zh_TW" just in case (although LANGUAGES=all` probably covers this. Start the game and look in the language settings for the Chinese, Japanese, and Korean entries.

@Leland

This comment has been minimized.

Show comment
Hide comment
@Leland

Leland Nov 5, 2017

Contributor

What's the order of your preferred languages in System Preferences > Language & Region? E.g:

screen shot 2017-08-23 at 8 01 19 pm

Contributor

Leland commented Nov 5, 2017

What's the order of your preferred languages in System Preferences > Language & Region? E.g:

screen shot 2017-08-23 at 8 01 19 pm

@BrettDong

This comment has been minimized.

Show comment
Hide comment
@BrettDong

BrettDong Nov 6, 2017

Contributor

Are you linking with ncursesw? I ran into a similar problem before because I linked with ncurses library without w, which doesn't render wide characters properly.

Contributor

BrettDong commented Nov 6, 2017

Are you linking with ncursesw? I ran into a similar problem before because I linked with ncurses library without w, which doesn't render wide characters properly.

@sxmichaels sxmichaels changed the title from Non-ASCII characters render with garbage symbols to [OSX] Non-ASCII characters render as garbage when default OS language is CJK Nov 6, 2017

@sxmichaels

This comment has been minimized.

Show comment
Hide comment
@sxmichaels

sxmichaels Nov 6, 2017

Contributor

@Leland Aha, that's a step in the right direction. My language order is:

  1. Simplified Chinese
  2. English
  3. Traditional Chinese

Switching English back to the top makes Cataclysm's non-ASCII text display properly, so the bug is likely related to the OS language setting. I've changed the bug title to reflect that. Based on this, I've discovered that that I can work around this issue by starting Cataclysm with LC_ALL=en_US.utf-8. I'm not sure what this implies about the source of the issue.

@BrettDong Looking back at my build output, the linker step is indeed called with `-lncurses. It looks like this comes from line 566 of the Makefile:

LDFLAGS += -lncurses

If I naively change that to -lncursesw, the non-ASCII chars still display as garbage, and other parts of the interface are also rendered improperly (chars misaligned, etc).

2017-11-06 11 30 09

Contributor

sxmichaels commented Nov 6, 2017

@Leland Aha, that's a step in the right direction. My language order is:

  1. Simplified Chinese
  2. English
  3. Traditional Chinese

Switching English back to the top makes Cataclysm's non-ASCII text display properly, so the bug is likely related to the OS language setting. I've changed the bug title to reflect that. Based on this, I've discovered that that I can work around this issue by starting Cataclysm with LC_ALL=en_US.utf-8. I'm not sure what this implies about the source of the issue.

@BrettDong Looking back at my build output, the linker step is indeed called with `-lncurses. It looks like this comes from line 566 of the Makefile:

LDFLAGS += -lncurses

If I naively change that to -lncursesw, the non-ASCII chars still display as garbage, and other parts of the interface are also rendered improperly (chars misaligned, etc).

2017-11-06 11 30 09

@BrettDong

This comment has been minimized.

Show comment
Hide comment
@BrettDong

BrettDong Nov 7, 2017

Contributor

When I was using Mac about a year ago, everything worked fine for me. I was using OS X 10.11 El Capitan at that time. I installed gettext by Homebrew too. One thing different is that I was using a "pure" Simplified Chinese system. I bought the MacBook Air in China and it runs a pre-installed Chinese version of Mac OS X. I didn't change anything about Language & Region in the system settings. So I suspect that the problem is related to multi-language on macOS.

Contributor

BrettDong commented Nov 7, 2017

When I was using Mac about a year ago, everything worked fine for me. I was using OS X 10.11 El Capitan at that time. I installed gettext by Homebrew too. One thing different is that I was using a "pure" Simplified Chinese system. I bought the MacBook Air in China and it runs a pre-installed Chinese version of Mac OS X. I didn't change anything about Language & Region in the system settings. So I suspect that the problem is related to multi-language on macOS.

@Leland

This comment has been minimized.

Show comment
Hide comment
@Leland

Leland Nov 7, 2017

Contributor

Yeah note #20665 – I'm pretty sure that it and this stem from the same underlying issue

Contributor

Leland commented Nov 7, 2017

Yeah note #20665 – I'm pretty sure that it and this stem from the same underlying issue

@OzoneH3

This comment has been minimized.

Show comment
Hide comment
@OzoneH3

OzoneH3 Dec 1, 2017

Member

Fixed with #22513

Member

OzoneH3 commented Dec 1, 2017

Fixed with #22513

@OzoneH3 OzoneH3 closed this Dec 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment