Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: semantic tokenizing of string escape/format characters #45753

Open
soypat opened this issue Apr 24, 2021 · 8 comments
Open

x/tools/gopls: semantic tokenizing of string escape/format characters #45753

soypat opened this issue Apr 24, 2021 · 8 comments
Assignees

Comments

@soypat
Copy link

@soypat soypat commented Apr 24, 2021

What version of Go are you using (go version)?

$ gopls version
golang.org/x/tools/gopls v0.6.10
    golang.org/x/tools/gopls@v0.6.10 h1:8Ebz8PymS2umcuCFhoz67unyJfWsipjTIrkBUF9kypg=

$ VsCode settings:
 "gopls": {	
        "ui.semanticTokens": true,
	},
    "editor.semanticHighlighting.enabled": true,

What did you do?

Write following code in VSCode with semanticTokens enabled

const httpResponse = "HTTP/1.0 200 OK\r\nContent-Type: text/html\r\nPragma: no-cache\r\n\r\n<h2>..::TinyGo Rocks::..</h2>"
fmt.Sprintf("%v", httpResponse)

What did you expect to see?

Semantic tokenization of \n\r\t escaped characters and format characters like %v in strings inside a Format function.

What did you see instead?

Monocolor string like the one github displays.

@gopherbot gopherbot added this to the Unreleased milestone Apr 24, 2021
@suzmue suzmue modified the milestones: Unreleased, gopls/unplanned Apr 26, 2021
@pjweinb
Copy link

@pjweinb pjweinb commented Apr 26, 2021

What should semantic tokens return in this case? (the possibilities are in the section on semantic tokens in https://microsoft.github.io/language-server-protocol/specifications/specification-3-17/)

@soypat
Copy link
Author

@soypat soypat commented Apr 26, 2021

Maybe an unused token, such as class could be used for such end. I'm not sure how to check what VScode uses for other languages, is there a way?

From here the following semantic tokens are available:

Semantic Tokens

  • type
  • class
  • enum
  • struct
  • typeParameter
  • parameter
  • variable
  • property
  • enumMember
  • event
  • function
  • method
  • macro
  • keyword
  • modifier
  • comment
  • string
  • number
  • regexp
  • operator

Semantic Token Modifiers

  • declaration
  • definition
  • readonly
  • static
  • deprecated
  • abstract
  • async
  • modification
  • documentation
  • defaultLibrary
@soypat
Copy link
Author

@soypat soypat commented Apr 27, 2021

Looks like the default coloring for these characters before enabling semantic tokenizing is property.

@pjweinb
Copy link

@pjweinb pjweinb commented Apr 28, 2021

This is more a vscode issue than a gopls issue, but here's the vscode story. Based on experiments I believe what happens is the following, to decide the foreground color:

  1. If there is a semantic token, use that color.
  2. If not, if there is a textmate label, use that color.
  3. Otherwise, use the default color.

In this case the semantic token is a string, so vscode uses the color for strings. If gopls didn't return string semantic tokens, then vscode would show the colors from the textmate grammar, which would show different colors for \n etc. In semantic tokens I know of no way to get different colors in the string (i tried all 10 of the SemanticTokenModifiers, and they all use the same string color.)

So there are two possiblities for gopls:

  1. Leave things the way they are. Then the vscode user's choice would be semantic tokens and drab strings, or turn off semantic tokens and get colorful strings.
  2. Don't return string semantic tokens. Then vscode users would get colorful strings, but other LSP clients would get incomplete semantic tokens. I do not know if there are any LSP clients using semantic tokens, other than vscode.
@pjweinb
Copy link

@pjweinb pjweinb commented Apr 28, 2021

See the comment in #45753.

There are only two choices, as I see it:

  1. Leave things the way they are. Then the vscode user's choice would be semantic tokens and drab numbers, or turn off semantic tokens and get colorful numbers.
  2. Don't return number semantic tokens. Then vscode users would get colorful numbers, but other LSP clients would get incomplete semantic tokens. I do not know if there are any LSP clients using semantic tokens, other than vscode.
@soypat
Copy link
Author

@soypat soypat commented Apr 28, 2021

See the comment in #45753.

There are only two choices, as I see it:

1. Leave things the way they are. Then the vscode user's choice would be semantic tokens and drab numbers, or turn off semantic tokens and get colorful numbers.

2. Don't return number semantic tokens. Then vscode users would get colorful numbers, but other LSP clients would get incomplete semantic tokens. I do not know if there are any LSP clients using semantic tokens, other than vscode.

@pjweinb I think this comment was maybe meant for #45792?

@pjweinb
Copy link

@pjweinb pjweinb commented Apr 28, 2021

@soypat
Copy link
Author

@soypat soypat commented Apr 29, 2021

Well I don't believe it is reasonable to be picky about what is implemented from the LSP. Option 1 seems like the better of two options. That said, is there any chance this could be brought in as an optional parameter to gopls? Something that can be modified in settings.json in VSCode for example.

@pjweinb pjweinb self-assigned this Apr 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants