Some keys are not recognized correctly when using a Japanese keyboard

[landonepps]

Keybindings for certain keys do not work as intended with a Japanese keyboard on macOS. When defining keybindings, the @ key (right of 'P') and '[' key are both recognized as a [. Shift+@ is recognized as shift+`. The ¥ key is 'unrecognized' and shift+¥ is recognized as shift+.

There may be issues with other keys, too. I noticed this one specifically because there is no ` key on a Japanese keyboard and it's used to open the terminal.

I'm using my MacBook's default keyboard which is Japanese. I also have a bluetooth Apple Japanese keyboard that experiences the same issue.

• VSCode Version: 1.10.1
• OS Version: macOS 10.12.4

Steps to Reproduce:

1. Open 'Preferences -> Keyboard Shortcuts' (cmd+k cmd+s)
2. Click 'Define Keybinding' (cmd+k cmd+k)
3. Press @ or ¥ on a Japanese keyboard.

Here's a picture of the keyboard layout for reference:

[landonepps]

I'd be willing to work on this, if someone more experienced with the codebase could point me in the right direction.

[alexdima]

@landonepps

Quite a lot have changed in the upcoming 1.11 regarding keyboard layouts and keybinding handling. I would suggest to first try to use the Insiders Build and see how it behaves there.

I have written a wiki page documenting how keybindings work in VS Code (1.11 - on the insiders channel) at https://github.com/Microsoft/vscode/wiki/Keybindings

Please give that a quick read and try to see if the wiki page provides sufficient information for you to understand what's going on. For Japanese, I believe the issue is in node-native-keymap where the OSX specific methods return a null pointer at one point. I would very much welcome a second pair of eyes on that code.

[landonepps]

You're right. Calling keymap.getCurrentKeyboardLayout() with a Japanese keyboard returns an empty object. I'll have a look and see if I can figure anything out.

[landonepps]

So I changed keyboard_mac.mm in node-native-keymap from

  TISInputSourceRef source = TISCopyCurrentKeyboardInputSource();
  CFDataRef layout_data = static_cast<CFDataRef>((TISGetInputSourceProperty(source, kTISPropertyUnicodeKeyLayoutData)));
  if (!layout_data) {
    // https://developer.apple.com/library/mac/documentation/TextFonts/Reference/TextInputSourcesReference/#//apple_ref/c/func/TISGetInputSourceProperty
    args.GetReturnValue().Set(result);
    return;
  }

to

  TISInputSourceRef source = TISCopyCurrentKeyboardInputSource();
  CFDataRef layout_data = static_cast<CFDataRef>((TISGetInputSourceProperty(source, kTISPropertyUnicodeKeyLayoutData)));
  if (!layout_data) {
    // TISGetInputSourceProperty returns null with a Japanese keyboard layout.
    // Using TISCopyCurrentKeyboardLayoutInputSource as a workaround.
    source = TISCopyCurrentKeyboardLayoutInputSource();
    layout_data = static_cast<CFDataRef>((TISGetInputSourceProperty(source, kTISPropertyUnicodeKeyLayoutData)));
    if (!layout_data) {
      // https://developer.apple.com/library/mac/documentation/TextFonts/Reference/TextInputSourcesReference/#//apple_ref/c/func/TISGetInputSourceProperty
      args.GetReturnValue().Set(result);
      return;
    }
  }

which is apparently a workaround for the NULL return value (see this stackoverflow page).

When using the modified node-native-keymap the ¥ key is still unrecognized, but the @ key now shows as "[BracketLeft]" with the @ sign displayed beneath. It used to display "[" when trying to bind the key.

[landonepps]

I created a PR for node-native-keymap
microsoft/node-native-keymap#8

[alexdima]

Brilliant!

[landonepps]

This issue isn't quite fixed. I can now use the @ key, but the _ and ¥ keys still show as unknown.
They appear as

  IntlRo:
   { value: '_',
     valueIsDeadKey: false,
     withShift: '_',
     withShiftIsDeadKey: false,
     withAltGr: '`',
     withAltGrIsDeadKey: true,
     withShiftAltGr: '`',
     withShiftAltGrIsDeadKey: false },
  IntlYen:
   { value: '¥',
     valueIsDeadKey: false,
     withShift: '|',
     withShiftIsDeadKey: false,
     withAltGr: '\\',
     withAltGrIsDeadKey: false,
     withShiftAltGr: '|',
     withShiftAltGrIsDeadKey: false },

in the results from getKeyMap(), so I'm not quite sure why @ now works and the others don't.

[alexdima]

@landonepps Thanks again for your work! ❤️

IntlRo and IntlYen should work fine after 362212b

Unfortunately I cannot test them as I only have access to ANSI and ISO keyboards, and not to JIS ones.

[landonepps]

I can confirm this has been completely fixed as of 1.12.1! IntlRo and IntlYen also work.

[richardkazuomiller]

Strange. I've had issues particularly with using [ and ] keys to switch tabs on some machines and not others. Today I rebooted my computer (I probably updated VSCode in the process) and started having problems in 1.12.1 that I didn't have before. shift+cmd+] and shift+cmd+[ don't work but shift+cmd+@ and shift+cmd+[ do what the expected shortcuts are supposed to respectively, just as the OP stated.

On another machine, it was always like that and I just got used to it, but then I switched to a new computer (the one I just updated) and keyboard where all the buttons worked as expected and was happy. Now I am sad.

I tried to fix it by defining custom keybindings, and for some reason @ is recognized as [, [ is recognized as ] and ] as \.

The attached image is what I see when I push cmd+shift+]. Why don't I see things like [BracketLeft]?

[richardkazuomiller]

To clarify, if I type @, [, or ] in an editor, they work. It's just keyboard shortcuts that are wonky.

[alexdima]

@richardkazuomiller

Let's check you have something similar to the OP:

• What physical keyboard are you working on? Is it an ANSI, ISO or JIS keyboard? I can try finding some resources online to help you identify it if you don't know what I'm talking about.
• Are you using the laptop keyboard or a keyboard attached to the laptop. If you're using an external keyboard, are the built-in keyboard and the external keyboard of the same type: ANSI, ISO, JIS.
• If you use an external keyboard that is not Apple branded, did you configure it correctly with OSX?
• What keyboard layout do you use in OSX?
• What keyboard layout do we detect if you open an editor and press F1 > Developer: Inspect Key mappings. Could you save that to a file and attach it here?

[richardkazuomiller]

I'm using a JIS keyboard on a Mac Mini, specifically this keyboard. As far as I know it's configured correctly, haven't had any other problems with it until today. Inspect Key mappings says:

{
	"id": "com.google.inputmethod.Japanese.Roman",
	"lang": "en"
}

[landonepps]

Does it work if you use the mac's default kotoeri IME instead of the google one?

[richardkazuomiller]

Same result for all input methods (Google Alphanumeric, Google Hiragana, Kotoeri Romaji, Kotoeri Hiragana).

[richardkazuomiller]

I also added US in OSX preferences and tried that for good measure, same result of those keys being switched.

[richardkazuomiller]

keymappings.txt

Sorry, didn't attach the file

[richardkazuomiller]

My current workaround is doing this:

  {
    "key": "shift+cmd+]",
    "command": "workbench.action.previousEditor"
  },
  {
    "key": "shift+cmd+\\",
    "command": "workbench.action.nextEditor"
  },
  {
    "key": "cmd+]",
    "command": "editor.action.outdentLines"
  },
  {
    "key": "cmd+\\",
    "command": "editor.action.indentLines"
  }

[alexdima]

@richardkazuomiller Thank you for getting back to us.

I think this might be an issue within our C++ code that reads the keyboard layout from the OSX - https://github.com/Microsoft/node-native-keymap

The C++ module works by reading the current keyboard layout, and then proceeds to use OSX APIs to convert all scan code + modifier combinations to produced characters. In this case (by looking at the keybindings.txt you attached), the module reports that @ is produced only via Shift+[Digit2]. i.e.

"Digit2": {
	"value": "2",
	"valueIsDeadKey": false,
	"withShift": "@",
	"withShiftIsDeadKey": false,
	"withAltGr": "™",
	"withAltGrIsDeadKey": false,
	"withShiftAltGr": "€",
	"withShiftAltGrIsDeadKey": false
}

Looking at the picture this looks incorrect. It appears you have a scan code that produces @ directly. I am sorry I cannot investigate this properly, because I only have access in the office to ANSI or ISO keyboards. And it appears OSX behaves differently based on using a JIS keyboard or not. If you would like to continue the investigation, it is very easy to get the node module up and running:

• pre-requisites: node.js
• clone from https://github.com/Microsoft/node-native-keymap.git
• npm install
• npm run test

This will output the mappings that we use in VS Code. And perhaps you can spot something fishy in that code or debug and see what's going on. Thank you!

[richardkazuomiller]

I haven't had time to look at any C++ code yet, but another weird thing started happening.

I have two projects open. In one window, the keyboard shortcuts I made work, the other they get overridden because VSCode started to recognize the keyboard properly. I've attached a screenshot.

Why does this happen in some windows and not others and how just make it do one or the other?

Strangely, the keymapping for Digit2 has also changed to " in the window in which I took the screenshot. I've also attached the keymappings for that window.

keymappings2.txt

[richardkazuomiller]

So weird ... I closed and reopened the window that was recognizing they keyboard input wrong at the time I first commented on this issue, and it start working. Same keyboard, same process (haven't exited VSCode since I posted this), just different window. If I figure out how to reproduce the issue someday I guess I'll come back then.

[alexdima]

@richardkazuomiller This is most weird. Are you perhaps switching between two different keyboard layouts and between two different physical keyboards? The C++ methods we're using have the ...last... word in them. So that might make a difference, i.e. what was the last physical keyboard that was used before that code executes... I'm sorry I'm not an expert either in the OSX APIs.