Flycheck Errors are misaligned when lines contain multibyte characters

[TOTBWF]

Thank you for the bug report

• I am using the latest version of lsp-mode related packages.
• I checked FAQ and Troubleshooting sections
• You may also try reproduce the issue using clean environment using the following command: M-x lsp-start-plain

Bug description

When a line contains a multibyte character, the flycheck overlays are misplaced. This is because of the following lines in sp-diagnostics--flycheck-start:

               (flycheck-error-new
                :buffer (current-buffer)
                :checker checker
                :filename buffer-file-name
                :message message
                :level (lsp-diagnostics--flycheck-calculate-level severity? tags?)
                :id code?
                :group source?
                :line (lsp-translate-line (1+ start-line))
                :column (1+ (lsp-translate-column start-character))
                :end-line (lsp-translate-line (1+ end-line))
                :end-column (1+ (lsp-translate-column end-character)))))

In particular, the spec states thatend-character and start-character will be measured in code units, whereas we are treating them as measuring grapheme clusters. These sorts of problems pervade the codebase, but this instance is particular egregious.

Steps to reproduce

Add multi-byte characters to a file, and use an LSP server that correctly follows the spec. If the server is truly spec compliant it will use UTF-16, which will break lsp-mode even further, but this also manifests when using UTF-8.

Expected behavior

I would expect that the underline is placed correctly.

Which Language Server did you use?

A custom server that I am authoring.

OS

MacOS

Error callstack

No response

Anything else?

No response