Completion breaks with multibyte characters

[arp242]

The following file:

package x

import (
    "fmt"
    "net/mail"
)

func t() {
    var m mail.Address
    fmt.Printf("%s → %s\n", m.)
}

Completing the m doesn't work in lsc. I tried VSCode, and it works there, so it seems a problem in vim-lsc rather than gopls.

What it sends to gopls is:

{
  "id": 2,
  "jsonrpc": "2.0",
  "method": "textDocument/completion",
  "params": {
    "position": {
      "character": 29,
      "line": 9
    },
    "textDocument": {
      "uri": "file:///tmp/tgo_20210620_w6ByRcao/main.go"
    }
  }
}

Putting the cursor right on top of the m and starting completion does work. Replacing that → (U+2192) with e.g. > also fixes it. So it seems it should send "character": 27 instead of "character": 29.

Looking at the LSC specification, character offsets are kinda weird:

The offsets are based on a UTF-16 string representation. So a string of the form a𐐀b the character offset of the character a is 0, the character offset of 𐐀 is 1 and the character offset of b is 3 since 𐐀 is represented using two code units in UTF-16. To ensure that both client and server split the string into the same line representation the protocol specifies the following end-of-line sequences: ‘\n’, ‘\r\n’ and ‘\r’.

Microsoft's love for UTF-16 strikes again 🙃

Right now it looks like it just sends the byte offset(?)

[arp242]

Sorry, I forgot to mention I'm using the latest master (4b0fc48)

I don't know what you comparability requirements are for vim-lsc, but Vim somewhat recently added some multibyte-aware functions to make all of this easier, e.g.:

:echo charidx(getline('.'), 29)
27

This requires Vim 8.2.2233 though (Dec 2020), and it'll be quite some time before e.g. Debian people will get this, but you can use e.g. has() to check for it. Older Vims can get the broken behaviour, new Vims the new behaviour.

Also see charcol().

I couldn't really figure out the logic in completion.vim right now or I'd send a patch 😅 But with some luck it should be a comparatively simple fix(?)

[natebosch]

@arp242 - Thanks for the pointer to charcol!

Could you try the multibyte-position branch and confirm that it solves your issue? From my debugging it seems like it should, but I think the server I'm running against might have a complimentary bug.

[arp242]

Thanks! It seems improved, but it doesn't display all completions:

I checked the LSP logs, and the request and responses are completely identical for the two examples (x and →), so it seems this part is correct, but somehow something else is still causing problems? 🤔

I tested with a clean vimrc as well.

Logs for completeness sake (but I don't think this is where the problem is):

{
  "id": 2,
  "jsonrpc": "2.0",
  "method": "textDocument/completion",
  "params": {
    "position": {"character": 26, "line": 11},
    "textDocument": {"uri": "file:///tmp/tgo_20210624_hs3UF4yn/main.go"}
  }
}

{
  "id": 2,
  "jsonrpc": "2.0",
  "result": {
    "isIncomplete": true,
    "items": [
      {
        "detail": "string",
        "documentation": "user@domain\n",
        "filterText": "Address",
        "insertTextFormat": 1,
        "kind": 5,
        "label": "Address",
        "labelDetails": {},
        "preselect": true,
        "sortText": "00000",
        "textEdit": {
          "newText": "Address",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      },
      {
        "detail": "string",
        "documentation": "Proper name; may be empty.\n",
        "filterText": "Name",
        "insertTextFormat": 1,
        "kind": 5,
        "label": "Name",
        "labelDetails": {},
        "sortText": "00001",
        "textEdit": {
          "newText": "Name",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      },
      {
        "detail": "func() string",
        "documentation": "String formats [..].\n",
        "filterText": "String",
        "insertTextFormat": 1,
        "kind": 2,
        "label": "String",
        "labelDetails": {},
        "sortText": "00002",
        "textEdit": {
          "newText": "String",
          "range": {"end": {"character": 26, "line": 11}, "start": {"character": 26, "line": 11}}
        }
      }
    ]
  }
}