Add emoji support to Windows Console

[HLFH]

Please support emoji within Windows Console.

Very useful when you code in vim newsletters for startups or when you categorize stuff by emoji.

---

Maintainer notes:

Input

Beta Give feedback

1. COOKED_READ (cmd.exe) doesn't properly support emoji input #1503
   Area-CookedRead Area-Input In-PR Issue-Feature Needs-Tag-Fix Priority-1 Product-Conhost +7
2. [Epic] COOKED READ and character conversion fun with all our different Convert* functions #7777
   Area-CookedRead Area-Input Impact-Correctness Issue-Bug Issue-Scenario Needs-Tag-Fix Product-Conhost Product-Conpty +8
   

Output

Beta Give feedback

1. [Epic] Text Buffer rewrite #8000
   Area-Output Area-Server Issue-Scenario Needs-Tag-Fix Product-Conhost Product-Terminal +6
2. Feature Request: Finish full unicode support (M:N cell rendering, ZWJ?) #1472
   Area-Rendering Issue-Feature Product-Conhost Product-Terminal +4
   

[zadjii-msft]

Already on the backlog :)

[mikemaccana]

Sweet! Another use case: I have a command line app that outputs warnings using ⚠.

[mqudsi]

@zadjii-msft will this include support for non-latin unicode characters? i.e. can Arabic or Japanese characters not found in the currently deployed console font be displayed with a different font face?

[zadjii-msft]

Hypothetically, yes.

Arabic is kinda it's own problem though - there's pretty much no chance we'll be implementing left to right language support any time soon, bt the characters might be able to render correctly.

@adiviness can speak more on the topic if there's more to share.

[adiviness]

Arabic is a difficult one because of the additional support required for bidirectional text and ligatures. As far as I’m aware not many terminal emulators currently support it.

There is also a difference between the console being able to render emoji and supporting font fallback in the case of the current font not having a glyph. They are both items on our backlog.

[SidShetye]

+1 Our build and CI scripts have emojis for success ✔️, warning ⚠ and errors ❌ for quick, productive eyeballing of logs.

[miniksa]

Can we please not +1 issues? Please use reactions or subscribe to notifications.

[SidShetye]

@miniksa , I wouldn’t get too distracted by the +1. You could entirely ignore it. The main purpose of typing it out was to articulate the use case. Presumably platform/product owners ought to be very interested in customer use cases ... thought granted in this case it’s not too extravagant.

[bitcrazed]

@SidShetye - appreciate your comment - your use-case is not unusual, but is useful to hear about.

In general, we ask people not to +1 because we'd like to avoid people +1-ing (esp. with no further comment), which just ends up as noise, and makes threads more difficult to parse and manage.

Sharing additional comments, context, observations, issues, etc. is FAR more valuable than a +1 ;)

[luxalpa]

Weird that nobody mentioned it, but the Yarn package manager uses Emoji's and it's a bit annoying that they are only displayed as squares :/

[bitcrazed]

Thanks @destructive-dragon - there are many tools that contain/emit emoji, but the Console is not yet able to render them.

[kavdev]

@bitcrazed You mentioned in this twitter thread on the conpty release that we still need to wait for a new buffer and a new renderer (DirectWrite). Are those the only two major blockers left?

[bitcrazed]

@kavdev Essentially, yes. To display emoji glyphs, we first have to be able to store (potentially compound) Unicode code-points for each glyph (E.g. Ninjacats), but we also have to be able to render them, which requires font-fallback, which GDI doesn't support.

We'll be working on improvements to both the Console's text buffer implementation, and also the Renderer in future releases.

[paul-at]

Is this still on the backlog? Unicode 13 was released in March 2020...

Seems to work for me at the moment in Windows Terminal Preview 1.12.3472.0 in both PS 5.1 and WSL2/Ubuntu, although #1472 bug (in a form of a broken Unicode sequence when using backspace) is still there.

[brupelo]

Hey guys, is this the official thread about emoji support?

I've just tried Version: 1.15.2874.0 running a simple python script that uses rich:

from rich._emoji_codes import EMOJI
from rich.console import Console

if __name__ == "__main__":
    console = Console()
    for k, v in EMOJI.items():
        console.print(f":{k}:")

And everything has rendered correctly on win10... Problem comes if I try to add emojis manually by pressing win+.,
in that case I'll just get an incorrect render, ie:

Anyway, is this one already reported? Is this the right backlog work-item to suscribe to? I've read another ones pinging to this one so I'm assuming it's the "official" one... 🙄

Evolving really nicely the software btw, congrats guys!

[zadjii-msft]

To clarify:

• Outputting emoji to the Terminal generally works quite well today. Especially in an environment like WSL, where everything is already in UTF-8.
  • There's a long tail of bugs remaining here. Some sizing issues.
  • Then, there's also the case of emoji made from m a n y codepoints, which we're tracking in [Epic] Text Buffer rewrite  #8000 (more or less).
• Inputting emoji really what's missing here. That's the visible part of "trying to type an emoji into CMD.exe". We're largely tracking that in [Epic] COOKED READ and character conversion fun with all our different Convert* functions #7777, and the many threads linked to that.

This has kinda become the mega thread for both, though, most of the remaining work remains in #7777.

[brupelo]

@zadjii-msft Hey Mike, thanks, cool stuff. So I guess the issue happening when copy/pasting emojis from other software such as SublimeText and not being rendered properly is also a related work-item from this one, ie:

You can see how in SublimeText the emojis are rendered properly, similar in the good old clipbrd.exe software but when i copy/paste into wt the render will be broken.

You can find the emojis.txt file attached
here

[zadjii-msft]

Yea, basically what you're seeing is the Input version there. Pasting emoji into cmd.exe is akin to just typing an emoji at the prompt, and cmd.exe does a bad/nonexistent job of rendering those. Just outputting that file straight to the Terminal (after a chcp 65001) does better (though, there's a LOT of emoji in that file leveraging ZWJ's, so those doen't behave super well. That's the output side of things).

[o-sdn-o]

It seems to me that cmd.exe does not play any role here. Between cmd.exe and Windows Terminal there is conhost.exe (openconsole.exe), it is responsible for such handling of emoji. If we replace conhost.exe (openconsole.exe) with something of our own, then there will be the expected result

Terminal.Dev.2023-01-03.19-41-56.mp4

[zadjii-msft]

Sorry, I omitted details for the sake of brevity. Cmd.exe uses the COOKED_READ input handling provided by the Console API, and the server for that API is conhost. So yes, patching openconsole (=== conhost.exe) will fix this, and that's what the whole collection of issues is about.

[marklit]

Will there be support for country flags at some point?

[TheDecryptor]

That's outside of Terminal's control, the Segoe UI Emoji font just doesn't include flags.

#2664 would allow customising the fonts Terminal tries to use, you could then include something like "Noto Color Emoji", which does include graphical flags.

[marklit]

Has anyone here tried to edit the Segoe UI Emoji font file and add in / replace the country flag emojis from another font? The Terminal doesn't support images so printing out GIFs or SVGs is a non-starter.

[Welding-Torch]

When will we get Nerd Font Icons(Glyphs) support in conhost.exe?

[DHowett]

When will we get Nerd Font Icons(Glyphs) support in conhost.exe?

You should be able to install a "nerd font" font and set it as your font today. Also last year, and perhaps some years prior to that one.