[LSP] Escaped characters in strings are not highlighted anymore

[acoutts]

Describe the bug
Previously, escaped characters within strings were highlighted in a different color (I believe light yellow), to make them more obvious. But after activating LSP this doesn't seem to happen anymore.

To Reproduce

final str = 'this string \nhas an escaped character in it';

Expected behavior
Escaped characters should be more visible.

Screenshots

Versions (please complete the following information):

• VS Code version:

1.57.0-insider
Commit: b95b73b7b074f5dddccd57e5e290617f1a767ec1

• Dart extension version: 3.22.0
• Dart/Flutter SDK version:

Flutter 2.3.0-0.1.pre • channel unknown • unknown source
Framework • revision d72bfb8d07 (12 days ago) • 2021-04-26 06:05:55 -0700
Engine • revision de6e1adf97
Tools • Dart 2.14.0 (build 2.14.0-18.0.dev)

[DanTup]

@bwilkerson is there any existing way to find the escaped characters within a string (it's a SimpleStringLiteral at the point highlights/semantic tokens are produced, but it seems to only have the raw string value)? The non-LSP textmate grammar finds these just using regex (though I'm not certain it was completely reliable).

[bwilkerson]

Yes. Given a SimpleStringLiteral named string

• The expression string.value will return the content of the string with delimiters removed and escapes converted to their actual character values.
• The expression string.literal.lexeme will return the unprocessed string literal including delimiters and the raw string marker (r) if there is one.

If you want to find escape sequences yourself you'll need to use the second expression and duplicate some of the logic from pkg/_fe_analyzer_shared/lib/src/parser/quote.dart:136.

[DanTup]

Perfect, thanks! I've had a go at this at https://dart-review.googlesource.com/c/sdk/+/199483 (the parser code didn't seem directly reusable, although the rules seemed pretty simple).

[DanTup]

This is fixed, but will require both an updated SDK and updated Dart-Code to apply (server changes mark the escape characters and Dart-Code maps it to the textmate constant.character.escape to get the colour from the theme - though you could set this manually in your settings too).

The original version was actually very naive and didn't consider hex/unicode characters correctly, it always just highlighted one character:

The new one should behave correctly for \x and \u.

(the misplaced squiggles are unrelated - I've filed that at dart-lang/sdk#46005).