Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support emoji presentation sequences #41

Merged
merged 1 commit into from
Apr 23, 2024
Merged

Support emoji presentation sequences #41

merged 1 commit into from
Apr 23, 2024

Conversation

Jules-Bertholet
Copy link
Contributor

@Jules-Bertholet Jules-Bertholet commented Apr 23, 2024

#35 take 2.


UAX11 says:

[UTS51] emoji presentation sequences behave as though they were East Asian Wide, regardless of their assigned East_Asian_Width property value.

Lookup is done with a 2-level trie.


In terms of UTS 51 conformance, with this PR, this crate will give the correct widths for:

However, it may overestimate (though never underestimate) the rendered widths of:


The not-yet-released Unicode 16 adds 8 new non-emoji standardized variation sequences that affect width: https://unicode.org/alloc/Pipeline.html#variation_sequences, https://www.unicode.org/L2/L2023/23212r-quotes-svs-proposal.pdf. In time, we'll need to support those as well.

@Manishearth Manishearth merged commit 73f816e into unicode-rs:master Apr 23, 2024
2 checks passed
@Jules-Bertholet Jules-Bertholet deleted the emoji-presentation branch April 23, 2024 17:13
@Advait-M
Copy link

Advait-M commented Apr 26, 2024

Hey 👋 ! I'm an engineer from Warp. So awesome to see this landed just 3 days ago - I was just digging deep into a fish-related Starship prompt issue which ultimately led me to Emoji_Presentation with Unicode for the ☁️ glyph (and related glyphs), so this is a wild coincidence haha 🔥 (I opened fish-shell/fish-shell#10461 just yesterday!). We use unicode-width to help calculate widths within our terminal grid, so this will help a bunch (we gotta fix up our widths + rendering logic).

Confirmed that the changes above help get the correct width of 2 for \u{2601}\u{FE0F} which maps to ☁️ with Emoji_Presentation via variation selector (as opposed to the previous result of 1) 🙌 !

Was curious when we expect this change will be included as part of a release cc @Manishearth perhaps? We're currently on version 0.1 which we can update to 0.1.11 but I believe this change isn't in a release cut yet, which makes sense given recency. We can also temporarily point to a commit directly as well, though ideally we'd avoid that for production Warp 😅 - just wanted to get a sense of the usual release cadence/cycle here?

And thanks a ton for working on this @Jules-Bertholet ! 🎉

@Manishearth
Copy link
Member

Published 0.1.12

I will warn you though: UAX 11 is not, and will never be, 100% accurate for monospace terminal size computation. "monospace" is not a concept that consistently makes sense across Unicode, and crates making strong assumptions about that should rethink their model.

@Advait-M
Copy link

Published 0.1.12

I will warn you though: UAX 11 is not, and will never be, 100% accurate for monospace terminal size computation. "monospace" is not a concept that consistently makes sense across Unicode, and crates making strong assumptions about that should rethink their model.

Awesome, thanks!

Ah gotcha - ack, thanks for the note! Yeah, we'll definitely have some fallback behavior that ensures it's not a fully broken experience in any case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants