[SR-13807] Use UTF-8 on Windows

[swift-ci]

Previous ID	SR-13807
Radar	None
Original Reporter	svanimpe (JIRA User)
Type	Bug
Status	Closed
Resolution	Done

Environment

29/10 snapshot on Windows

Additional Detail from JIRA

Votes	1
Component/s	Standard Library
Labels	Bug, Windows
Assignee	@compnerd
Priority	Medium

md5: 206fbad8245b125d62b0d21d1914e7ff

Issue Description:

Whenever I run a Swift app on Windows, I first have to set the correct code page in my terminal with chcp 65001, or add the following to my app:

import WinSDK
_ = SetConsoleOutputCP(UINT(CP_UTF8))

Is there a way to do this by default? @compnerd suggested doing it in the runtime.

[swift-ci]

Comment by Steven Van Impe (JIRA)

I noticed that setting the code page is not enough to support Unicode in the Command Prompt and PowerShell applications. Certain characters show up, but many emojis show up as question marks instead.

This does work (mostly) when using the new Windows Terminal.

[swift-ci]

Comment by Steven Van Impe (JIRA)

I've noticed another issue that may be related.

The output of this code has some issues, even in Windows terminal:

import WinSDK
_ = SetConsoleOutputCP(UINT(CP_UTF8))
print("�‍♂️��‍���������")

The male face palm emoji shows up as a female face palm emoji, followed by a male gender sign. The other emojis are seemingly followed by other (invisible) characters as well, as there are spaces in the output.

Basic emojis work fine.

[EDIT: The emojis in my example displayed when I posted this, but now there seems to be an encoding issue here as well, as I'm posting this from Windows. To reproduce this issue, you can use the male face palm emoji]

[compnerd]

stdlib: change the console to UTF-8 on start by compnerd · Pull Request #40632 · apple/swift (github.com)

[swift-ci]

Comment by Steven Van Impe (JIRA)

Will this be in 5.5.3?

[compnerd]

This hasn't been backported to 5.5.3. I suppose that it is a nice enough improvement, windows specific and pretty self-contained. I'll see if I can backport it.

[compnerd]

This is fixed on main. Backporting to 5.5.3, while interesting, is not tracked by this issue.