autoClosingPairs notIn functionnality

[PyvesB]

Hello,

When I'm writing a comment or a string, I don't want a pair to be inserted when typing characters such as apostrophes for instance, especially as this moves my cursor around. Similarly, when I'm in the middle of writing a string but for some reason I have deleted the end delimiter, I want the string to be closed with a unique delimiter character rather than a pair being inserted. The current behaviour is often frustrating.

The language configuration does support more advanced cleverness for these delimiters thanks to the notIn parameter:

{
  "autoClosingPairs": [
...
    { "open": "'", "close": "'", "notIn": ["string", "comment"] },
...
  ]
}

tm4e already has support to parse this configuration: https://github.com/eclipse/tm4e/blob/f52ef37e230bd339df145a0b1bf8883e7f67af4c/org.eclipse.tm4e.languageconfiguration/src/main/java/org/eclipse/tm4e/languageconfiguration/internal/LanguageConfiguration.java#L133

However, I don't believe it's currently being used anywhere.

The comment case is quite easy to fix, as the language configuration defines a comments section which tells you what comments look like in a given language. There is a handy existing method that indicates whether you're in a comment or not: https://github.com/eclipse/tm4e/blob/8731eb78c1e4a612cdec5df42c7b407126e2d089/org.eclipse.tm4e.languageconfiguration/src/main/java/org/eclipse/tm4e/languageconfiguration/internal/supports/CommentSupport.java#L25 A few extra lines of code LanguageConfigurationAutoEditStrategy.java would probably do the trick.

Nevertheless, the string case is more challenging: there doesn't seem to be anything in the configuration that tells you what a string looks like in that language. I'm failing to see how we could implement a isInString equivalent based on the language configuration.

Any thoughts?

Cheers,

Pyves

[angelozerr]

I'm failing to see how we could implement a isInString equivalent based on the language configuration.

notIn is a feature that I had had no time to implement. If I remember vscode uses the the textmate tokens to see you are in string or not.

[PyvesB]

notIn is a feature that I had had no time to implement. If I remember vscode uses the the textmate tokens to see you are in string or not.

The tokens are information contained in the grammar if I'm not mistaken? The big issue here is that the feature that implements the character insertion pair logic is org.eclipse.tm4e.languageconfiguration, and as far as I can tell it has no awareness of the grammar.

[angelozerr]

The tokens are information contained in the grammar if I'm not mistaken?

Exactly!

The big issue here is that the feature that implements the character insertion pair logic is org.eclipse.tm4e.languageconfiguration, and as far as I can tell it has no awareness of the grammar.

It's a big issue and if we want to do like vscode, language configuration should get token information from the textmate grammar, it's a very hard issue because tokens could not be available when notIn should be executed since token are computed in async mode (to avoid freezing the editor).

See my old comments #166 (comment)

[PyvesB]

It's a big issue and if we want to do like vscode, language configuration should get token information from the textmate grammar, it's a very hard issue because tokens could not be available when notIn should be executed since token are computed in async mode (to avoid freezing the editor).

See my old comments #166 (comment)

Thanks for explaining, I've got a better grasp of some of the complexities now.

Having thought about this some more, I don't think there is any good alternative to getting token information from the TextMate grammar. I previously mentioned that there is an isInComment method relying on the comments parameter of the language configuration, but having looked at its implementation it doesn't quite do the right thing in all cases, for example when the comment delimiter is contained within a string or an end block delimiter is escaped. Maybe wordPattern could help us fix some of the edge cases, but even then we haven't started solving the more complex notIn string case.

[sebthom]

Implemented with #407