Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unicode combining characters rendering issues #1172

Closed
ntessore opened this issue Feb 21, 2021 · 3 comments · Fixed by #1356
Closed

unicode combining characters rendering issues #1172

ntessore opened this issue Feb 21, 2021 · 3 comments · Fixed by #1356
Labels
Renderer Text rendering issues, including CoreText renderer
Milestone

Comments

@ntessore
Copy link

ntessore commented Feb 21, 2021

Describe the bug

At some point during the last couple of updates, an issue appeared with rendering unicode combining characters, in particular diacritics/accents. Take the combination x⃗ [U+0078 U+20d7] which now renders as in screenshot 1 (note the x is in column 1). Turning off the Core Text Renderer restores the correct rendering as in screenshot 2.

Edit: It works fine in snapshot 166.

To Reproduce
Copy x⃗ from here and paste it into MacVim.

Expected behavior
Render x⃗.

Screenshots

MacVim default:

image

Core Text Renderer disabled (or using snapshot 166):

image

Environment (please complete the following information):
macOS 11

$ mvim --version
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Dec 19 2020 21:23:05)
macOS version
Included patches: 1-2164
Compiled by root@Mac-1608412769109.local
Huge version with MacVim GUI.  Features included (+) or not (-):
+acl               +file_in_path      -mouse_sysmouse    -tag_old_static
+arabic            +find_in_path      +mouse_urxvt       -tag_any_white
+autocmd           +float             +mouse_xterm       -tcl
+autochdir         +folding           +multi_byte        +termguicolors
-autoservername    -footer            +multi_lang        +terminal
+balloon_eval      +fork()            -mzscheme          +terminfo
+balloon_eval_term +fullscreen        +netbeans_intg     +termresponse
+browse            +gettext           +num64             +textobjects
++builtin_terms    -hangul_input      +odbeditor         +textprop
+byte_offset       +iconv             +packages          +timers
+channel           +insert_expand     +path_extra        +title
+cindent           +ipv6              +perl/dyn          +toolbar
+clientserver      +job               +persistent_undo   +transparency
+clipboard         +jumplist          +popupwin          +user_commands
+cmdline_compl     +keymap            +postscript        +vartabs
+cmdline_hist      +lambda            +printer           +vertsplit
+cmdline_info      +langmap           +profile           +virtualedit
+comments          +libcall           +python/dyn        +visual
+conceal           +linebreak         +python3/dyn       +visualextra
+cryptv            +lispindent        +quickfix          +viminfo
+cscope            +listcmds          +reltime           +vreplace
+cursorbind        +localmap          +rightleft         +wildignore
+cursorshape       +lua/dyn           +ruby/dyn          +wildmenu
+dialog_con_gui    +menu              +scrollbind        +windows
+diff              +mksession         +signs             +writebackup
+digraphs          +modify_fname      +smartindent       -X11
+dnd               +mouse             -sound             -xfontset
-ebcdic            +mouseshape        +spell             +xim
+emacs_tags        +mouse_dec         +startuptime       -xpm
+eval              -mouse_gpm         +statusline        -xsmp
+ex_extra          -mouse_jsbterm     -sun_workshop      -xterm_clipboard
+extra_search      +mouse_netterm     +syntax            -xterm_save
-farsi             +mouse_sgr         +tag_binary        
   system vimrc file: "$VIM/vimrc"
     user vimrc file: "$HOME/.vimrc"
 2nd user vimrc file: "~/.vim/vimrc"
      user exrc file: "$HOME/.exrc"
  system gvimrc file: "$VIM/gvimrc"
    user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
       defaults file: "$VIMRUNTIME/defaults.vim"
    system menu file: "$VIMRUNTIME/menu.vim"
  fall-back for $VIM: "/Applications/MacVim.app/Contents/Resources/vim"
Compilation: clang -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_MACVIM -Wall -Wno-unknown-pragmas -pipe  -DMACOS_X -DMACOS_X_DARWIN  -g -O2 -arch x86_64 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1 -Wall -Wextra -Wshadow -Werror -Wno-error=missing-field-initializers -Wno-error=deprecated-declarations -Wno-error=unused-function       
Linking: clang   -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/readline/lib -L. -fstack-protector-strong -L/usr/local/lib -L/usr/local/opt/libyaml/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/readline/lib  -arch x86_64 -L/usr/local/lib -o Vim        -lm  -lncurses -liconv /usr/local/lib/libintl.a -framework AppKit   -fstack-protector  -L/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE 

Additional context
Use a font that can render the glyph.
See below.

@ychin
Copy link
Member

ychin commented Mar 1, 2021

I can reproduce this but need to look into what a valid fix is.

@ychin ychin added the Renderer Text rendering issues, including CoreText renderer label Mar 1, 2021
@ychin ychin added this to the snapshot-171 milestone Mar 8, 2021
@ychin ychin modified the milestones: snapshot-171, snapshot-172 Apr 1, 2021
@ntessore
Copy link
Author

ntessore commented Jul 9, 2022

Is it possible that this only occurs when the selected guifont does not support the combining character, and a fallback is used?

Compare Menlo which falls back to a different x with the vector arrow:

image

And JuliaMono which has support for the vector arrow:

image

@ntessore
Copy link
Author

@ichizok Could this be related to #680?

@ychin ychin modified the milestones: Release 174, Release 175 Sep 12, 2022
ychin added a commit to ychin/macvim that referenced this issue Jan 21, 2023
MacVim previously didn't really render composing characters with
multiple glyphs correctly. For simple ones like 'â' it would work fine
because Core Text just generates one glyph for it, but for more
complicated ones like 'x⃗' the renderer didn't properly query the advance
of the glyphs to put them at the correct position. Refactor the logic to
keep track of the current cell/glyphs and make sure to track the
advances fo the glyphs so the composing chars' glyphs would be laid out
properly on top of the cell.

We need to be careful with the tracking, because in Vim we force the
text rendering to be monospaced, and so we maintain the tracking within
a single grapheme (which can consist of multiple glyphs), but when we
move to the next grapheme we reset the position so we can get proper
monospaced rendering even for non-monospaced texts like CJK or stylized
texts.

Fix macvim-dev#995
Fix macvim-dev#1172
ychin added a commit to ychin/macvim that referenced this issue Jan 21, 2023
MacVim previously didn't really render composing characters with
multiple glyphs correctly. For simple ones like 'â' it would work fine
because Core Text just generates one glyph for it, but for more
complicated ones like 'x⃗' the renderer didn't properly query the advance
of the glyphs to put them at the correct position. Refactor the logic to
keep track of the current cell/glyphs and make sure to track the
advances fo the glyphs so the composing chars' glyphs would be laid out
properly on top of the cell.

We need to be careful with the tracking, because in Vim we force the
text rendering to be monospaced, and so we maintain the tracking within
a single grapheme (which can consist of multiple glyphs), but when we
move to the next grapheme we reset the position so we can get proper
monospaced rendering even for non-monospaced texts like CJK or stylized
texts.

Fix macvim-dev#995
Fix macvim-dev#1172
ychin added a commit to ychin/macvim that referenced this issue Jan 29, 2023
MacVim previously didn't really render composing characters with
multiple glyphs correctly. For simple ones like 'â' it would work fine
because Core Text just generates one glyph for it, but for more
complicated ones like 'x⃗' the renderer didn't properly query the advance
of the glyphs to put them at the correct position. Refactor the logic to
keep track of the current cell/glyphs and make sure to track the
advances fo the glyphs so the composing chars' glyphs would be laid out
properly on top of the cell.

We need to be careful with the tracking, because in Vim we force the
text rendering to be monospaced, and so we maintain the tracking within
a single grapheme (which can consist of multiple glyphs), but when we
move to the next grapheme we reset the position so we can get proper
monospaced rendering even for non-monospaced texts like CJK or stylized
texts.

Fix macvim-dev#995
Fix macvim-dev#1172
ychin added a commit to ychin/macvim that referenced this issue Jan 30, 2023
MacVim previously didn't really render composing characters with
multiple glyphs correctly. For simple ones like 'â' it would work fine
because Core Text just generates one glyph for it, but for more
complicated ones like 'x⃗' the renderer didn't properly query the advance
of the glyphs to put them at the correct position. Refactor the logic to
keep track of the current cell/glyphs and make sure to track the
advances fo the glyphs so the composing chars' glyphs would be laid out
properly on top of the cell.

We need to be careful with the tracking, because in Vim we force the
text rendering to be monospaced, and so we maintain the tracking within
a single grapheme (which can consist of multiple glyphs), but when we
move to the next grapheme we reset the position so we can get proper
monospaced rendering even for non-monospaced texts like CJK or stylized
texts.

Fix macvim-dev#995
Fix macvim-dev#1172
ychin added a commit that referenced this issue Feb 7, 2023
Updated to Vim 9.0.1276

Features
====================

Dictionary lookup
--------------------

You can now use Force Touch or Cmd-Ctrl-D to look up definitions of word
under the cursor (or selected text in visual mode). This will also
preview URLs, and support data types such as phone numbers and
addresses. #1312 #1313

This feature can also be invoked programmatically from VimScript (see
`:h macvim-lookup`). #1315

Tool bar / Touch Bar / menu icons
--------------------

You can now use SF Symbols for Tool bar and Touch Bar icons, including
using different symbol styles such as "palette" or "multicolor". Menu
items can now also use the `icon=` syntax to specify icons as well. See
`:help macvim-toolbar-icon` for details. #1329

The default tool bar also has updated icons to look similar to SF
Symbols used by newer macOS versions. #1214 by @sfsam

Window management actions
--------------------

There are new `macaction`'s for managing the MacVim window. The new
`zoomLeft`/`zoomRight` actions allow you to pin the window to the
left/right of the screen, and there are also new actions for interacting
with Stage Manager (requires macOS 13+). See `:h macvim-actions` for
details. #1330

Pre-release updates / Sparkle 2
--------------------

MacVim now supports pre-release software builds. It's sometimes hard for
us to release frequent updates due to the desire to pick a stable
upstream Vim version, needing to test the release on multiple OS
versions, making sure there aren't half-complete or buggy features, and
other reasons.

This new feature now allows us to push pre-release beta builds out in a
more frequent fashion, which could be useful if there are particular
features or fixes that you would like to try out before the next
official release. Pre-release builds will be released depending on bug
fixes and features instead of a fixed cadence. Do note that these
pre-release builds may not be as well-validated and may have half-baked
features.

If you are using the built-in auto-updater to update MacVim, you can
turn this on by going to Advanced settings pane, and enable "Enable
pre-release software updates".

This feature is only available for macOS 10.13 or above.

The auto-updater has also been updated from Sparkle 1.27.1 to 2.3.0 for
10.13+ builds. Legacy (10.9-10.12) builds are still using Sparkle 1.

See #1332.

New Vim features
--------------------

New `smoothscroll` option allows you to scroll through a long wrapped
line (using Ctrl-E or mouse wheel) without immediately jumping to the
next line. (v9.0.0640)

`splitscroll` option has been renamed `splitkeep`, with more flexibility
than before. (v9.0.0647)

Sound playback on macOS is now supported. You can use `has('sound')` to
check. See `help sound` for details. (v9.0.0694)

Terminals now support `:confirm` for `:q`, etc, which also means
MacVim's Cmd-W will work properly for terminal windows. (v9.0.0710)

Virtual text had numerous bugs fixed.

General
====================

Legacy build for 10.9 - 10.12
--------------------

Per a previous announcement (#1271), the default MacVim binary will now
require macOS 10.13 or above. Users of macOS 10.9 - 10.12 can use a
separate "legacy" build which will still be supported. The legacy binary
will still have the latest versions of Vim and be supported, but may not
have all the latest features (e.g. pre-release builds).

If you are using the auto-updater (Sparkle) to update MacVim, it should
"just work" and find the best version for you. If you are downloading
MacVim from the website, there is also a link to download the legacy
version marked for 10.9+ as well. If you download the normal binary
marked for 10.13+ from the website, it won't work on these older macOS
versions.

See #1331.

Fixes
====================

CoreText Renderer clipping and rendering bugs
--------------------

Unicode characters with multiple composing characters (e.g. "x⃗") will
now render correctly. #1172

Texts (e.g. Tibetan, Zalgo texts) that are taller than the line height
will no longer be clipped inappropriately. You can use a new setting
`MMRendererClipToRow` to re-enable clipping if the tall texts are
distracting. #995 / #1356

Tab crash
--------------------

Fixed a crash when opening new tabs that seems to only occur in macOS 13
Ventura. #1333

Other bugs
--------------------

- Fixed non-native full screen not working well with the notch on newer
  MacBook's when set to not show menu bar. You can also use
  `MMNonNativeFullScreenSafeAreaBehavior` to force MacVim to use the
  notch area as well if you don't mind some content being obscured. Note
  that the previous release also claimed it fixed this, but because the
  binary was built against an old macOS SDK (Big Sur), the fix did not
  work in the binary release. #1261
- Allow "Open untitled window: never" and "After last window closes:
  Quit MacVim" to be set together again. Added safeguards to make sure
  doing so won't immediately close the app. #1338
- Edit.Cut / Copy menu items will now be properly disabled when there
  isn't selected text. #1308
- Fixed potential `:emenu` crash when the menu is associated with an
  action in a non-valid mode. #1305
- Fixed bug where just bringing up the right-click (or the
  MacVim→Services) menu would somehow copy the selected texts to the
  system clipboard. #1300
- Fixed a Japanese input method bug where using left/right arrow to move
  to a different section of the input text would previously result in
  the candidate list not showing up at the correct position. #1312
- Fix non-CoreText renderer not handling text styles like strikethrough
  correctly (note: this renderer has been deprecated for a while and you
  should not use it). #1296
- This release uses an older sh/bash syntax file because the latest one
  in Vim has a bug. #1358

Misc
====================

New settings:

- "No drop shadows" (Appearance). #1301
- "Treat Ctrl-click as right-click" (Input) (#1326). This was previously
  configurable via command-line, but now also possible in the settings
  pane under the new "Input" category.

"About MacVim" now reports the version number in a clearer way with
clearly specified release number vs Vim version.

Known Issues
====================

Printing
--------------------

Printing using File→Print or `:hardcopy` is currently not working under
macOS 13 Ventura due to its removal of PostScript support in the Preview
app. This will be fixed in a later release. See the issue for
workarounds. #1347

Scripting
====================

- Scripting languages versions:
    - Perl is now built against 5.30, up from 5.18.
    - Ruby is now built against 3.2, up from 3.1.

Compatibility
====================

Requires macOS 10.9 or above. (10.9 - 10.12 requires downloading a
separate legacy build)

Script interfaces have compatibility with these versions:

- Lua 5.4
- Perl 5.30
- Python2 2.7
- Python3 3.10
- Ruby 3.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Renderer Text rendering issues, including CoreText renderer
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants