feat(lsp): add lsp.util.split_lines and lsp.util.split_lines_iter

[dmitmel]

How do I generate documentation for lsp.util.split_lines and lsp.util.split_lines_iter?

[mjlbach]

in what scenario do we need the fallback?

[dmitmel]

It is simply written defensively, AFAIK it is impossible to set fileformat to an invalid value

[mjlbach]

I need to think more about these. get_buf_text is the only thing I think is 100% necessary or correct.

The issue with \r is internally, we can have \r without a properly tracked newline split. (so we can have midline \r unlike other editors)

[mjlbach]

Given we don't internally handle \r according to the specification, I'm not sure this is correct

[dmitmel]

Are you talking about the function or the usages of it?

[mjlbach]

I'm talking about we shouldn't be splitting lines on \r before sending to the server

[dmitmel]

@mjlbach Alright, I give up. I'll undo the changes to usages of lsp.util.split_lines, but let's at least move the function itself to a different PR. Don't you need something like that in order to handle in-line \rs?

[dmitmel]

Do any Language Servers return CRLF newlines in textEdits? Asking for whether vim.split(text_edit.newText, '\n', true) should use fileformat-dependent newline too.

[dmitmel]

but let's at least move the function itself to a different PR. Don't you need something like that in order to handle in-line \rs?

Nevermind, you can just nvim_buf_get_lines and split each line on \r. The only place where I imagine this function could be useful is for lsp.util.get_lines (and a similar one in lsp.diagnostic which can be refactored to use lsp.util.get_lines), which can't do fileformat detection. Please tell me if you need split_lines - if not, I am pulling it out.

[dmitmel]

@mjlbach What do we do with this PR? (Ref: #16277 (comment))

[mjlbach]

WDYM what do we do? I need to think about some of the thing's bjorn said about our endline handling, a lot of my internal justification for maintaining the line ending when communicating with the language server is undermined by the fact we save the buffer to disk as utf-8

[dmitmel]

Well, I mean, should I make it a PR with just a fix for #16231 and move split_lines to a different PR, or keep both in this PR, or something else? I have undone the changes to usage of split_lines because I couldn't find a server which reports back CR or CRLF line endings on hover, signature help, diagnostics or something of that matter, i.e. unrelated to file content tracking.

[dmitmel]

Moved the buf_get_full_text fix to #16334

[mfussenegger]

Closing this because it conflicts with #25272 and the functions added aren't used by other lsp components