lsp-lens-mode breaks end-of-visual-line

[Khady]

Describe the bug

With lsp-lens-mode enabled, when at the start of a line that follows a lens, end-of-visual-line does nothing. The point is not moved at the end of the line. end-of-line doesn't have this issue.

To Reproduce

type demo =
  [ `Ok
  | `Error
  | `Exn of string
  ]

let f zzz xxx = zzz + xxx

let f zzz xxx = zzz + f xxx zzz

let f = Format.std_formatter

M-x lsp-lens-mode RET M-g c 59 RET M-: (end-of-visual-line) RET

Expected behavior

end-of-visual-line should have the usual behavior, point should be moved at the end of the line.

Which Language Server did you use

ocamlmerlin-lsp

OS

debian strech, emacs 27 from git

Error callstack

>>> ocamlmerlin-lsp:30391(async)
Content-Length: 199

{
  "jsonrpc": "2.0",
  "method": "textDocument/codeLens",
  "params": {
    "textDocument": {
      "uri": "file:///tmp/test.ml"
    }
  },
  "id": 566
}

<<<< ocamlmerlin-lsp:30391
{
  "id": 566,
  "jsonrpc": "2.0",
  "result": [
    {
      "range": {
        "start": {
          "line": 10,
          "character": 0
        },
        "end": {
          "line": 10,
          "character": 28
        }
      },
      "command": {
        "title": "Format.formatter",
        "command": ""
      }
    },
    {
      "range": {
        "start": {
          "line": 8,
          "character": 0
        },
        "end": {
          "line": 8,
          "character": 31
        }
      },
      "command": {
        "title": "int -> int -> int",
        "command": ""
      }
    },
    {
      "range": {
        "start": {
          "line": 6,
          "character": 0
        },
        "end": {
          "line": 6,
          "character": 25
        }
      },
      "command": {
        "title": "int -> int -> int",
        "command": ""
      }
    }
  ]
}

[yyoncho]

It does not work only if you place the cursor at first char of the line. Here it is minimal code to reproduce the issue:

1. M-x eval-defun when placed on the opening bracket.
2. M-x end-of-visual-line - FAIL.

(let ((ov (make-overlay (point-at-bol) (point-at-eol))))
  (overlay-put ov 'before-string "XXX\n"))

This behaviour is either a emacs bug or the expected behavour when there is overlay with before-string set which contains \n.

(CC @Eli-Zaretskii)

[Eli-Zaretskii]

Lisp programs that futz with overlay strings with embedded newlines and then move visually should be prepared to deal with potential trouble. In this case, when you are at the beginning of the line, are you before the before-string or after it? If before, then the visual line ends at the newline embedded in the overlay string, right?

I made a change on the master branch to handle this case, hope I didn't break anything else by doing that. In general, Lisp programs are well advised to stay away of such techniques.

Thanks.

[yyoncho]

@Eli-Zaretskii

If before, then the visual line ends at the newline embedded in the overlay string, right?

I guess this was the problem.

Closing based on the previous comment.