Fix off-by-one error converting to LSP UTF8 offsets with multi-byte char

[krobelus]

On this file,

fn main() {
    let 된장 = 1;
}

when using "positionEncodings":["utf-16"] I get an "unused variable" diagnostic on the variable
name (codepoint offset range 8..10). So far so good.

When using positionEncodings":["utf-8"], I expect to get the equivalent range in bytes (LSP:
"Character offsets count UTF-8 code units (e.g bytes)."), which is 8..14, because both
characters are 3 bytes in UTF-8. However I actually get 10..14.

Looks like this is because we accidentally treat a 1-based index as an offset value: when
converting from our internal char-indices to LSP byte offsets, we look at one character to many.
This causes wrong results if the extra character is a multi-byte one, such as when computing
the start coordinate of 된장.

Fix that by actually passing an offset. While at it, fix the variable name of the line number,
which is not an offset (yet).

Originally reported at kakoune-lsp/kakoune-lsp#740

[krobelus]

Also, when trying to work around this, I found that if the client sends positionEncodings=["utf-16", "utf-8"], we still pick Utf8:

    for enc in client_encodings {
        if enc == &PositionEncodingKind::UTF8 {
            return PositionEncoding::Utf8;
        } else if enc == &PositionEncodingKind::UTF32 {
            return PositionEncoding::Wide(WideEncoding::Utf32);
        }
        // NB: intentionally prefer just about anything else to utf-16.
    }

    PositionEncoding::Wide(WideEncoding::Utf16)

why not pick the client's preferred encoding here? (i.e. whichever of UTF8/UTF16/UTF32 comes first)

[Veykril]

I believe the reasoning is that we need to do less work for utf8, given rust natively handles utf8 strings.

[krobelus]

Right, LSP doesn't specify (what a surprise) whether the client preference or server preference should win.
So the best workaround is for the client to send only ["utf16"] in that case.
Fortunately this is irrelevant usually.

[Veykril]

Thanks!
@bors r+

[bors]

📌 Commit d24b0ba has been approved by Veykril

It is now in the queue for this repository.

[bors]

⌛ Testing commit d24b0ba with merge 8e581ac...

[bors]

☀️ Test successful - checks-actions
Approved by: Veykril
Pushing 8e581ac to master...

[BenjaminBrienen]

Right, LSP doesn't specify (what a surprise) whether the client preference or server preference should win.
So the best workaround is for the client to send only ["utf16"] in that case.
Fortunately this is irrelevant usually.

Isn't this the exact kind of "why" that would be useful to have as a code comment?