Rendering Issues with zh_CN.UTF-8

[greycodee]

Hello, when I run the example on mac, the typesetting is out of order, what's the matter?

• My system is macOS Big Sur 11.4
• Terminal is oh-my-zsh

[meowgorithm]

Hi! I’m not able to reproduce this on any of my Macs. I suspect your terminal may not be using UTF-8 (see #30).

Would you mind running the locale command and pasting the output here?

Also, to be thorough, what terminal emulator and font are you using? It appears to be Lucida Console in macOS stock terminal, though it would be good to verify this.

[greycodee]

Hi! I’m not able to reproduce this on any of my Macs. I suspect your terminal may not be using UTF-8 (see #30).

Would you mind running the locale command and pasting the output here?

Also, to be thorough, what terminal emulator and font are you using? It appears to be Lucida Console in macOS stock terminal, though it would be good to verify this.

locale:

╰─$ locale
LANG="zh_CN.UTF-8"
LC_COLLATE="zh_CN.UTF-8"
LC_CTYPE="zh_CN.UTF-8"
LC_MESSAGES="zh_CN.UTF-8"
LC_MONETARY="zh_CN.UTF-8"
LC_NUMERIC="zh_CN.UTF-8"
LC_TIME="zh_CN.UTF-8"
LC_ALL=

[greycodee]

my terminal is iTerm2, and the font I use is Monaco Regular

[greycodee]

I solved this issue based on this answer，
The solution is：export LC_CTYPE="en_US.UTF-8"

[meowgorithm]

Okay, that’s good to know. Still, it would be good to figure out why things aren’t rendering properly under zh_CN.UTF-8. If you don't mind I'm going to leave this issue open for now so we can hopefully solve for this.

[greycodee]

Okay, that’s good to know. Still, it would be good to figure out why things aren’t rendering properly under zh_CN.UTF-8. If you don't mind I'm going to leave this issue open for now so we can hopefully solve for this.

Ok, I'll leave that question open

[meowgorithm]

Thanks! Related: mattn/go-runewidth#14

[meowgorithm]

Also for reference: setting LC_CTYPE=zh_CN.UTF-8 recreates the behavior. Setting LC_CTYPE=zh_CN.UTF-8 RUNEWIDTH_EASTASIAN=0 corrects it.

[greycodee]

Also for reference: setting LC_CTYPE=zh_CN.UTF-8 recreates the behavior. Setting LC_CTYPE=zh_CN.UTF-8 RUNEWIDTH_EASTASIAN=0 corrects it.

This also works. When I set export RUNEWIDTH_EASTASIAN=0, it is normal.

[yechentide]

LC_CTYPE="ja_JP.UTF-8" also causes incorrect rendering.
It can also be solved by export LC_CTYPE="en_US.UTF-8" or export RUNEWIDTH_EASTASIAN=0

[Qrnbth]

@meowgorithm
How to force terminal to run with this certain enviroment without configuring environment manually?
I tried

os.Setenv("RUNEWIDTH_EASTASIAN", "true")
os.Setenv("LC_CTYPE", "en_US.UTF-8")

before executing the main app, but that didn't work, any nice solution?

[ghostsquad]

@Qrnbth have you tried os.Setenv("RUNEWIDTH_EASTASIAN", "0") ?

[ghostsquad]

Encodings like this are very difficult to deal with because there's technically no correct answer to how something should be configured. For such a common failure as this, it makes sense to include a link in the --help of the application and in the readme. Basically just in as many places as the user may be looking at for assistance.

Now that this is known, it may also be worth while to pre-emptively check the value of that environment variable and see if there's a way to provide a warning to the user.

This isn't much different to other tools that, as an example, require a nerd font, in order to correctly display various icons and such.

[Ice-Mel]

@meowgorithm How to force terminal to run with this certain enviroment without configuring environment manually? I tried

os.Setenv("RUNEWIDTH_EASTASIAN", "true")
os.Setenv("LC_CTYPE", "en_US.UTF-8")

before executing the main app, but that didn't work, any nice solution?

In the source code of github.com/mattn/go-runewidth, it obtains the RUNEWIDTH_EASTASIAN environment variable using the init() function.

According to the execution order of init() in Golang, you can create a separate conf package in your project, and set the RUNEWIDTH_EASTASIAN environment variable in the init() function within this package. Then, you can import the conf package in the main package.

Just ensure that the conf package is the first in the import list of the main package, and it will take effect.

package conf

import (
	"os"
)

func init() {
	os.Setenv("RUNEWIDTH_EASTASIAN", "0")
}

package main

import (
	_ "project/conf"
	"github.com/charmbracelet/bubbles/viewport"
	tea "github.com/charmbracelet/bubbletea"
	"github.com/charmbracelet/glamour"
	"github.com/charmbracelet/lipgloss"
)

[Qrnbth]

@meowgorithm How to force terminal to run with this certain enviroment without configuring environment manually? I tried

os.Setenv("RUNEWIDTH_EASTASIAN", "true")
os.Setenv("LC_CTYPE", "en_US.UTF-8")

before executing the main app, but that didn't work, any nice solution?

In the source code of github.com/mattn/go-runewidth, it obtains the RUNEWIDTH_EASTASIAN environment variable using the init() function.

According to the execution order of init() in Golang, you can create a separate conf package in your project, and set the RUNEWIDTH_EASTASIAN environment variable in the init() function within this package. Then, you can import the conf package in the main package.

Just ensure that the conf package is the first in the import list of the main package, and it will take effect.

package conf

import (
	"os"
)

func init() {
	os.Setenv("RUNEWIDTH_EASTASIAN", "0")
}

package main

import (
	_ "project/conf"
	"github.com/charmbracelet/bubbles/viewport"
	tea "github.com/charmbracelet/bubbletea"
	"github.com/charmbracelet/glamour"
	"github.com/charmbracelet/lipgloss"
)

This do works,thanks