missing highlight on some non-ASCII identifiers

[yaohaizh]

From @nobodxbodon on March 5, 2019 18:43

Hi,

the differencies are in Line 4 between the English and Chinese versions as below:

Calculator class:

class Calculator {

  public static void main(String[] args) {
    Calculator calculator = new Calculator();
  }
}

Identifers in Chinese:

class 计算器类 {

  public static void main(String[] args) {
    计算器类 计算器 = new 计算器类();
  }
}

I'm using vscode 1.31.1 with Java Extension Pack 0.6.0 under Mac.

Thanks.

Copied from original issue: microsoft/vscode-java-pack#109

[yaohaizh]

From @nobodxbodon on March 5, 2019 18:57

One additional test shows the method argument shows different too:

  public void test1(Calculator calculator) {
    calculator.test1(this);
  }

Chinese version:

  public void 测试1(计算器类 计算器) {
    计算器.测试1(this);
  }

[fbricon]

This is an issue with the textmate based grammar from upstream VS Code, itself forked from Atom's.

[nobodxbodon]

@fbricon thanks for the diagnose. May I ask if you refer to the [A-Za-z$_] part in section like this? Would you suggest to move this issue to VS Code or Atom?

[fbricon]

This is partially fixed with the new semantic highlighting support in vscode-java 0.61.0. Requires "java.semanticHighlighting.enabled": true in settings.json. Without semantic highlighting, the Textmate grammar is still broken.