Characters with diacritics cannot be typed

[mifko]

• VSCode Version: 1.30.0
• OS Version: macOS Mojave 10.14.1 (18B75)

Steps to Reproduce:

1. Open integrated terminal in the VSCode
2. Ensure Slovak keyboard is select in the macOS
3. Try typing a character with diacritics (e.g.: č ť š or ž)

Result: Nothing appears in the terminal
Expected: Character with terminal appears in the terminal

More info on characters with diacritics:

• more languages use these, e.g. Slovak, Czech, Polish, ...
• on MacOS with Slovak keyboard selected, those are usually typed with a single keyboard hit: for instance, the same key that is on English keyboard used for typing a number (e.g. 2 5 or 7; the buttons in the top row of the keyboard) is used for characters with diacritics on Slovak keyboard

Additional info:

• this used to work few versions/months ago, unfortunately I haven't noticed exactly when it started
• it does work when terminal.app is started directly (the app outside of VSCode)
• characters with diacritics can be typed within the text editor itself, it's only integrated terminal where they do now work
• those characters cannot even be pasted from the clipboard to the integrated terminal
• characters without diacritics (e.g.: c t s or z) can be freely typed into the terminal

Does this issue occur when all extensions are disabled?: Yes

[Tyriar]

Duplicate #35901

those characters cannot even be pasted from the clipboard to the integrated terminal

If this is true then there is likely a problem with how text encoding is setup in your terminal.

[mifko]

Hi @Tyriar

Thanks for having a look at the issue even though I do not agree with marking this as a duplicate of #35901 - that one is an issue with cutting text due to font size/font family/line spacing, ...

My issue has nothing to do with how text is displayed in the terminal, it really is just an issue with actual typing into the terminal. When I write a simple node.js application that just logs a text to the console (see below) and execute it, then the characters with diacritics are displayed - no issue at all.

console.log('ščťž')

The problem is that I simply cannot type those characters within the terminal itself.

If this is true then there is likely a problem with how text encoding is setup in your terminal.

Could you maybe elaborate on this little bit more?

How can I change the way my integrated terminal is setup for the text encoding? Because as I mentioned in the issue, I cannot type those characters only into the integrated terminal within VSCode. I can successfully type the mentioned characters when I run terminal app directly, outside of VSCode.

So to me it looks like the integrated terminal is somehow "filtering" which keys can be pressed within itself and is allowing some but blocking/ignoring others.

But maybe there is some setting I don't know that I could use to configure "text encoding" for the integrated terminal, as you mention?

One more thing which I have just discovered:

• the typing doesn't work, when bash is running within the integrated terminal (but works when bash is executed in the terminal app outside of VSCode)
• BUT, when i start node (thus Node REPL) in the integrated, I can suddenly type those characters freely in the Node REPL. Then I quit Node REPL, which switches back to the bash, typing of the mentioned characters doesn't work again

So I cannot type those only within the bash running within the VSCode integrated terminal.

Any ideas what I can check / try / configure?

Many thanks!

[Tyriar]

the typing doesn't work, when bash is running within the integrated terminal (but works when bash is executed in the terminal app outside of VSCode)

This is further proof of my theory 😃

Could you echo $LANG and echo $LANGUAGE in both vscode's terminal and an external one and paste them here?

[mifko]

Hi

Output of echo $LANG

• from terminal (where everything works): sk_SK.UTF-8
• from integrated terminal (where the issue occurs): sk.UTF-8
• from node.js (using console.log(process.env.LANG)): sk.UTF-8

Output of echo $LANGUAGE

• from terminal (where everything works): empty string
• from integrated terminal (where the issue occurs): empty string
• from node.js (using console.log(process.env.LANGUAGE)): undefined

Then I tried setting the LANG environment variable to the value from terminal.app (LANG=sk_SK.UTF-8) and tried to type some characters with diacritics but still no luck.

Any other ideas maybe?

Many thanks,
Michal

[abnersn]

I can confirm this is also happening in Portuguese. I can type characters like áãé all over the editor and side bar, but it doesn't work on the integrated terminal.

• OS: Fedora 30 (fresh install)
• VSCode Version: 1.36.1 (snap package)
• Keyboard Layout: Default Brazilian Portuguese (br-abnt2).
• Language and enconding settings: Everything is pt_BR.UTF-8 (OS terminal, tty, and integrated terminal), which is correct.

Please bear in mind that some foreign keyboard layouts, like mine, have dedicated dead keys for special characters, we do not use Alt-Gr or other shortcuts to type non-ASCII characters like a typical US layout user would do. For example, to write á, we type the ' dead key, then the letter a. Only the letter a shows up if I do this on VSCode.

[abnersn]

Duplicate #35901

those characters cannot even be pasted from the clipboard to the integrated terminal

If this is true then there is likely a problem with how text encoding is setup in your terminal.

In my case, I can paste from the clipboard. IMO this is not at all similar to the #35901 issue.

[Tyriar]

I think this is actually a duplicate of #59438, upstream issue is xtermjs/xterm.js#2151

/duplicate #59438

[mifko]

Hi @Tyriar

Thank you for looking at this. However I am not sure this is a duplicate as in my case it's related to any character that contains diacritics. Though it behaves strange only in bash, in node it's fine.

This is how it looks on my mac, maybe you can tell more based on it, whether it is a duplicate or not really:

Many thanks,
Michal

[Tyriar]

@mifko do you type them with altgr? That seems to be the underlying problem with that issue xtermjs/xterm.js#2151 (comment)

[mifko]

Hi @Tyriar

No, i type them with a single key press.

My keyboard is set to "Slovak - qwerty". When this keyboard is active, pressing for instance the key with 6 on it, which would output 6 on english keyboard, outputs ž (however not in integrated terminal, but anywhere else).

I tried now switching to zsh and it outputs something to the terminal though it still looks pretty useless. But maybe that helps? It looks like being typed two chars at the same time?

And this is how it looks in the Terminal.app, both with bash and zsh:

Maybe any of that helps?

Many thanks,
Michal

[Tyriar]

@mifko thanks for the info/gif. Does changing the setting terminal.integrated.shellArgs.osx to [] or ["-l"] make a difference?

cc @jerch, something related to character encoding?

[mifko]

Hi @Tyriar

Tried to set "terminal.integrated.shellArgs.osx": [], restarted integrated terminal (Kill Terminal, Create New Integrated Terminal), no change, still chars with diacritics did not work.

Then tried to set "terminal.integrated.shellArgs.osx": ["-l"], again restarted terminal, again no help.

Even tried to change renderer ("terminal.integrated.rendererType": "dom") and font family ("terminal.integrated.fontFamily": "monospace") but no help either.

Many thanks,
Michal

[jerch]

@Tyriar Yes there is some mixing with utf8 bytes going on: the shown <00c5><00a1> for š are the utf8 bytes (0xC5 0xA1) for that character. Not sure what the 00c5 thing means here, if that contains the "codepoint width" the backend prolly gets 0x00 0xc5 0x00 0xa1 with weird padding bytes instead of the the utf8 bytes 0xc5 0xa1. Further debugging the onData event and the final bytes sent to the pty by node-pty might give the answer.

[mifko]

Hi @Tyriar and @jerch

Many thanks both of you for digging into this. Not sure when exactly did it get fixed, but I have tried right now (VSCode 1.38.1) and it works!

Many thanks to you a to whoever have fixed.
I knew VSCode and people behind it are the best! ❤

Thanks again,
Michal