Write following code in VSCode with semanticTokens enabled
hexReg=0x1337bitmask=0b0110_1100google=1E100planck=2e-23// How to tokenize this?c=1+20i
What did you expect to see?
Semantic tokenization of 0x, 0b, E, and e numeric modifiers. This was colored before enabling LSP semantic tokenization. The coloring for these could be keyword (this was the default coloring I was accustomed to seeing before enabling semantic tokenizing). As a bonus maybe the 0 in octal notation could be colored for clarity?
I'm not sure how i (imaginary unit) should be tokenized. It seems to serve a different purpose as it is a number by itself. It would probably be helpful to color it though! Maybe property token for it?
What did you see instead?
The text was updated successfully, but these errors were encountered:
0x1337, 1e100 and 1+20i all are a single number, all should be one single token, and if you have colourization enabled I would not expect e.g. the e in 1e100 to be of a different colour (why would it be? It's part of the number and has no meaning by itself). Same for the + and the i in complex literals.
I could really do without the e and i colorization in 1e10i. That said, I would benefit from some sort of colorization of 0x and 0b in 0x1337 and 0b0101 since they have a fundamental impact to the interpretation of the digits. It would make surfing Register definitions, flags and bitmasks a more pleasant task.
i think the right way to do this is to use the new options
'noSemanticString' and 'noSemanticNumber' which disables semantic tokens
for strings and numbers, allowing the native colorization to show through,
(It's hard for users to get useful colors from the semantic token modifiers)