Eglot usually fails to start when using TRAMP on Windows with clangd #662

jimporter · 2021-04-05T01:26:53Z

I'm trying out eglot (latest from MELPA, currently 20210403.953) on a Windows system running emacs 27.2. When running M-x eglot on a TRAMP file (connected to a Linux system), I usually get an error like Wrong type argument: "inserted-chars 200" (the number in the message varies). Every once in a while, it works, although I'm not able to reproduce that reliably. I'm not sure it matters, but I'm attempting to use clangd here.

I've tried using the sshx and plinkx protocols and both fail in the same way. I've also tried with both the version of TRAMP that ships with emacs (2.4.5.27.2) and the latest version on GNU ELPA (2.5.0.3); still no difference.

M-x eglot-events-buffer sadly doesn't have much of interest:

[stderr] I[18:08:59.901] clangd version 10.0.0-4ubuntu1 
[stderr] I[18:08:59.902] PID: 189082
[stderr] I[18:08:59.902] Working directory: /home/jim
[stderr] I[18:08:59.902] argv[0]: clangd
[stderr] I[18:08:59.902] Starting LSP over stdin/stdout

However, the error backtraces might be more informative. Here's a log generated after calling M-x toggle-debug-on-error:

M-x toggle-debug-on-error

Debugger entered--Lisp error: (wrong-type-argument "inserted-chars 200")
  signal(wrong-type-argument ("inserted-chars 200"))
  tramp-signal-hook-function(wrong-type-argument ("inserted-chars 200"))
  signal(wrong-type-argument ("inserted-chars 200"))
  tramp-handle-insert-file-contents("/plinkx:server:/tmp/tramp.yIuCLY" nil nil nil nil)
  apply(tramp-handle-insert-file-contents ("/plinkx:server:/tmp/tramp.yIuCLY" nil nil nil nil))
  tramp-sh-file-name-handler(insert-file-contents "/plinkx:server:/tmp/tramp.yIuCLY" nil nil nil nil)
  apply(tramp-sh-file-name-handler insert-file-contents ("/plinkx:server:/tmp/tramp.yIuCLY" nil nil nil nil))
  tramp-file-name-handler(insert-file-contents "/plinkx:server:/tmp/tramp.yIuCLY" nil nil nil nil)
  insert-file-contents("/plinkx:server:/tmp/tramp.yIuCLY" nil nil nil nil)
  insert-file-contents-literally("/plinkx:server:/tmp/tramp.yIuCLY")
  tramp-sh-handle-make-process(:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  apply(tramp-sh-handle-make-process (:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t))
  tramp-sh-file-name-handler(make-process :name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  apply(tramp-sh-file-name-handler make-process (:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t))
  tramp-file-name-handler(make-process :name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  make-process(:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  #f(compiled-function () #<bytecode 0x182ba11>)()
  #f(compiled-function (cl--cnm conn slots) #<bytecode 0x18a9629>)(#f(compiled-function (&rest cnm-args) #<bytecode 0x18a959d>) #<eglot-lsp-server eglot-lsp-server-18682f8> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>)))
  apply(#f(compiled-function (cl--cnm conn slots) #<bytecode 0x18a9629>) #f(compiled-function (&rest cnm-args) #<bytecode 0x18a959d>) (#<eglot-lsp-server eglot-lsp-server-18682f8> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>))))
  #f(compiled-function (&rest args) #<bytecode 0x190b27d>)(#<eglot-lsp-server eglot-lsp-server-18682f8> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>)))
  apply(#f(compiled-function (&rest args) #<bytecode 0x190b27d>) #<eglot-lsp-server eglot-lsp-server-18682f8> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>)))
  initialize-instance(#<eglot-lsp-server eglot-lsp-server-18682f8> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>)))
  #f(compiled-function (class &rest slots) "Default constructor for CLASS `eieio-default-superclass'.\nSLOTS are the initialization slots used by `initialize-instance'.\nThis static method is called when an object is constructed.\nIt allocates the vector used to represent an EIEIO object, and then\ncalls `initialize-instance' on that object." #<bytecode 0x1397635>)(eglot-lsp-server :name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>))
  apply(#f(compiled-function (class &rest slots) "Default constructor for CLASS `eieio-default-superclass'.\nSLOTS are the initialization slots used by `initialize-instance'.\nThis static method is called when an object is constructed.\nIt allocates the vector used to represent an EIEIO object, and then\ncalls `initialize-instance' on that object." #<bytecode 0x1397635>) eglot-lsp-server (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>)))
  make-instance(eglot-lsp-server :name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x182ba11>))
  apply(make-instance eglot-lsp-server :name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x1726ce9>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x18a5fd1>) :on-shutdown eglot--on-shutdown (:process #f(compiled-function () #<bytecode 0x182ba11>)))
  eglot--connect(c++-mode (transient . "/plinkx:server:/home/jim/") eglot-lsp-server ("clangd"))
  eglot(c++-mode (transient . "/plinkx:server:/home/jim/") eglot-lsp-server ("clangd") t)
  funcall-interactively(eglot c++-mode (transient . "/plinkx:server:/home/jim/") eglot-lsp-server ("clangd") t)
  call-interactively(eglot record nil)
  command-execute(eglot record)
  execute-extended-command(nil "eglot" nil)
  funcall-interactively(execute-extended-command nil "eglot" nil)
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

Starting the debugger on insert-file-contents and manually stepping through gives some (possibly) more information about the call stack. In this particular case, I got the following slightly-different error: Not a Tramp file name: "c:/plinkx:server:/home/jim/":

M-x debug-on-entry RET insert-file-contents

* tramp-run-real-handler(expand-file-name ("c:/Users/Jim/AppData/Local/Temp/tramp.0ZyuEL.TvEQ9..." "/plinkx:server:/home/jim/"))
* tramp-file-name-handler(expand-file-name "c:/Users/Jim/AppData/Local/Temp/tramp.0ZyuEL.TvEQ9..." "/plinkx:server:/home/jim/")
* #<subr insert-file-contents>("c:/Users/Jim/AppData/Local/Temp/tramp.0ZyuEL.TvEQ9..." nil nil nil nil)
* apply(#<subr insert-file-contents> ("c:/Users/Jim/AppData/Local/Temp/tramp.0ZyuEL.TvEQ9..." nil nil nil nil))
* insert-file-contents("c:/Users/Jim/AppData/Local/Temp/tramp.0ZyuEL.TvEQ9..." nil nil nil nil)
  tramp-handle-insert-file-contents("/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil)
  apply(tramp-handle-insert-file-contents ("/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil))
  tramp-sh-file-name-handler(insert-file-contents "/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil)
  apply(tramp-sh-file-name-handler insert-file-contents ("/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil))
  tramp-file-name-handler(insert-file-contents "/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil)
  #<subr insert-file-contents>("/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil)
  apply(#<subr insert-file-contents> ("/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil))
* insert-file-contents("/plinkx:server:/tmp/tramp.TvEQ96" nil nil nil nil)
  insert-file-contents-literally("/plinkx:server:/tmp/tramp.TvEQ96")
  tramp-sh-handle-make-process(:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  apply(tramp-sh-handle-make-process (:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t))
  tramp-sh-file-name-handler(make-process :name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  apply(tramp-sh-file-name-handler make-process (:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t))
  tramp-file-name-handler(make-process :name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  make-process(:name "EGLOT (jim/c++-mode)" :command ("sh" "-c" "stty raw > /dev/null; clangd") :connection-type pipe :coding utf-8-emacs-unix :noquery t :stderr #<buffer *EGLOT (jim/c++-mode) stderr*> :file-handler t)
  #f(compiled-function () #<bytecode 0x24445c1>)()
  #f(compiled-function (cl--cnm conn slots) #<bytecode 0x23751c5>)(#f(compiled-function (&rest cnm-args) #<bytecode 0x24446a5>) #<eglot-lsp-server eglot-lsp-server-2444618> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>)))
  apply(#f(compiled-function (cl--cnm conn slots) #<bytecode 0x23751c5>) #f(compiled-function (&rest cnm-args) #<bytecode 0x24446a5>) (#<eglot-lsp-server eglot-lsp-server-2444618> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>))))
  #f(compiled-function (&rest args) #<bytecode 0x2444689>)(#<eglot-lsp-server eglot-lsp-server-2444618> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>)))
  apply(#f(compiled-function (&rest args) #<bytecode 0x2444689>) #<eglot-lsp-server eglot-lsp-server-2444618> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>)))
  initialize-instance(#<eglot-lsp-server eglot-lsp-server-2444618> (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>)))
  #f(compiled-function (class &rest slots) "Default constructor for CLASS `eieio-default-superclass'.\nSLOTS are the initialization slots used by `initialize-instance'.\nThis static method is called when an object is constructed.\nIt allocates the vector used to represent an EIEIO object, and then\ncalls `initialize-instance' on that object." #<bytecode 0x13bea29>)(eglot-lsp-server :name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>))
  apply(#f(compiled-function (class &rest slots) "Default constructor for CLASS `eieio-default-superclass'.\nSLOTS are the initialization slots used by `initialize-instance'.\nThis static method is called when an object is constructed.\nIt allocates the vector used to represent an EIEIO object, and then\ncalls `initialize-instance' on that object." #<bytecode 0x13bea29>) eglot-lsp-server (:name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>)))
  make-instance(eglot-lsp-server :name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown :process #f(compiled-function () #<bytecode 0x24445c1>))
  apply(make-instance eglot-lsp-server :name "EGLOT (jim/c++-mode)" :events-buffer-scrollback-size 2000000 :notification-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445dd>) :request-dispatcher #f(compiled-function (server method params) #<bytecode 0x24445f9>) :on-shutdown eglot--on-shutdown (:process #f(compiled-function () #<bytecode 0x24445c1>)))
  eglot--connect(c++-mode (transient . "/plinkx:server:/home/jim/") eglot-lsp-server ("clangd"))
  eglot(c++-mode (transient . "/plinkx:server:/home/jim/") eglot-lsp-server ("clangd") t)
  funcall-interactively(eglot c++-mode (transient . "/plinkx:server:/home/jim/") eglot-lsp-server ("clangd") t)
  call-interactively(eglot record nil)
  command-execute(eglot record)
  execute-extended-command(nil "eglot" "eglot")
  funcall-interactively(execute-extended-command nil "eglot" "eglot")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

It's possible this is a bug in TRAMP (although it's not a problem I've ever encountered with TRAMP before) or a limitation of Windows pipes as mentioned in the README (though I'd expect the pipe is being set up on the Linux system). I'm happy to provide more information or to test a potential patch, though I don't think I'd be able to author a patch on my own as my Elisp skills probably aren't up to the challenge. :/

The text was updated successfully, but these errors were encountered:

joaotavora · 2021-04-05T09:08:50Z

Eglot + TRAMP + Windows is more than likely to not work. This was documented by @bjc in the source code:

(defun eglot--cmd (contact)
  "Helper for `eglot--connect'."
  (if (file-remote-p default-directory)
      ;; TODO: this seems like a bug, although it’s everywhere. For
      ;; some reason, for remote connections only, over a pipe, we
      ;; need to turn off line buffering on the tty.
      ;;
      ;; Not only does this seem like there should be a better way,
      ;; but it almost certainly doesn’t work on non-unix systems.
      (list "sh" "-c"
            (string-join (cons "stty raw > /dev/null;"
                               (mapcar #'shell-quote-argument contact))
             " "))
    contact))

I don't have any particular expertise to offer here, sorry.

joaotavora · 2021-04-05T09:11:50Z

I marked this emacs-bug. My suggestion is to open a bug report to the Emacs bug tracker where you will find more knowledgeable people about this, particularly Michael Albinus.

jimporter · 2021-04-05T21:07:31Z

Hmm, I would have thought that comment only applied when connecting to a non-Unix system, not for the system Emacs is running on. You're probably right, however, that the issue lies somewhere in TRAMP/Emacs. I'll see if I can construct a reduced test case that doesn't rely on Eglot, since I imagine that'll make it easier to get this fixed in TRAMP/Emacs.

Of course, if someone with more TRAMP expertise than me (maybe @bjc?) wants to take a look at this here, I'm happy to help, even if it's just by providing more debug logs.

jimporter · 2021-04-09T04:20:16Z

Testing this further, I don't see this problem when using ccls, so there's something unusual that clangd is doing.

I did also notice that Flymake doesn't highlight errors and gets stuck in a "Wait" state when using eglot over TRAMP (on both Windows and Linux clients); I'd guess that's a Flymake issue, though I haven't been able to narrow it down yet.

joaotavora · 2021-04-09T08:35:35Z

I'd guess that's a Flymake issue,

It's likely not, but we'd need a minimal reproduction recipe to reproduce it.

jimporter · 2021-04-11T03:26:43Z

It's likely not, but we'd need a minimal reproduction recipe to reproduce it.

Once I get some time to dig into the Flymake issue, I'll file a new issue with steps to reproduce it.

As for this issue, I've narrowed the problem down to this hook in jsonrpc.el (or, more likely, a callee of this code): https://github.com/emacs-mirror/emacs/blob/59342f689eaa4839b0fc15351ae48b4f1074a6fc/lisp/jsonrpc.el#L384-L398

If I remove that hook, my issue described above goes away. I can run M-x eglot, it starts up ok, and using completion works as I'd expect. I'm not totally sure what the problem is, but given what the hook is doing and the fact that M-x eglot sometimes succeeds even with the hook, my guess is that there's a race condition somewhere in Emacs (maybe in TRAMP or insert-file-contents).

Since it looks like you wrote most of jsonrpc.el, I'm wondering if you have any thoughts here? If not, that's ok. I have steps to reproduce this in vanilla Emacs 28 now, so that should make reporting it to the Emacs bug tracker easier.

jimporter · 2021-04-13T01:43:01Z

This is probably not an issue with eglot specifically. I can reproduce it in a vanilla Emacs 28 snapshot (acquired here), with the following code:

(require 'jsonrpc)

(defun eglot-minimal ()
  (interactive)
  (make-instance
   'jsonrpc-process-connection
   :name "eglot-minimal"
   :process (lambda ()
              (make-process
               :name "eglot-minimal"
               :command '("sh" "-c" "stty raw > /dev/null; clangd")
               :connection-type 'pipe
               :coding 'utf-8-emacs-unix
               :noquery t
               :stderr (get-buffer-create "*eglot-minimal stderr*")
               :file-handler t))))

(This code is extracted/reduced from eglot--connect.) If I eval the above and then open a remote C++ file (using sshx from a Windows 10 client to a Linux server with clang version 10.0.0-4ubuntu1) and then call M-x eglot-minimal, I get the same error as above. It's possible this is a bug in jsonrpc.el, in TRAMP, or in insert-file-contents.

Since @Ergus mentions the same problem in #667, hopefully we can coordinate to figure out what our setups have in common so we have a better idea of where this needs to be fixed.

jimporter · 2021-04-13T02:05:00Z

Oh, just a couple more small notes:

If I replace clangd in the Elisp snippet above with ccls, then eglot-minimal doesn't throw an error. That's good news, since it means eglot-minimal reproduces the issue I was seeing where clangd fails but ccls succeeds.
If I replace clangd with nonexistent, it throws (roughly) the same error as before: Wrong type argument: "inserted-chars 30", and the stderr buffer shows: sh: 2: nonexistent: not found. This is interesting, since it means the problem isn't specifically clangd.

Looking at this more, when running ccls, I don't get a stderr buffer. Presumably, this is because ccls doesn't print anything to stderr, at least not on startup; I verified this by running ccls in my shell just to see what happens. clangd does write to stderr though (likewise with nonexistent, sort of). So it looks like this issue could be summed up as...

jsonrpc.el fails over TRAMP when the process writes to stderr (on startup?)

jimporter · 2021-04-13T05:20:40Z

For what it's worth, the following hack seems to work:

(require 'eglot)
(defun eglot--cmd (contact)
  (if (file-remote-p default-directory)
       (list "sh" "-c"
            (string-join
             (append (cons "stty raw > /dev/null;"
                           (mapcar #'shell-quote-argument contact))
                     '("2>" "/dev/null"))
             " "))
    contact))

That is, we do what eglot--cmd normally does, but also append 2> /dev/null to the command in the TRAMP case so that we ignore stderr. This will (hopefully) let me use clangd in the short term while we move forward on a fix in Emacs.

I'll give others a bit of time to chime in on this while I also think over whether I have anything to add, and then I'll file a bug against Emacs for this.

Ergus · 2021-04-13T07:23:12Z

I'll give others a bit of time to chime in on this while I also think over whether I have anything to add, and then I'll file a bug against Emacs for this.

Yes, this seems like an API wrong use or a Tramp issue. I think that @albinus could help with this in any case to clarify either if the issue is in eglot code or in Tramp and a but report will be needed.

albinus · 2021-04-13T08:02:29Z

stderr in remote make-process is flaky. So I'd prefer an Emacs bug with a minimal recipe.

jimporter · 2021-04-13T18:03:56Z

@albinus The shortest recipe I've been able to get is here. Evaling the Elisp in that comment in Emacs 28 and then doing the following reproduces the issue for me:

C-x C-f /sshx:server:~/path/to/file.cpp
M-x eglot-minimal

I don't think the Tramp method matters (I saw this with plinkx too). I also don't think you need clangd to be installed. It seems that anything that writes to stderr - including a nonexistent executable - will cause this problem.

I noticed this on a Windows client connecting to a Linux server (I don't have access to my Linux client at the moment to test there). @Ergus, were you seeing this on a different OS combination? If this happens on all platforms, that would make it easier to test. I'll file an Emacs bug once I'm sure of what the OS requirements are to reproduce this.

At least based on the results of the test in 05fe647, this only happens when you're actually connecting remotely. The "loopback" method in that test (which just invokes sh) apparently works fine.

joaotavora · 2021-04-13T18:08:45Z

@albinus The shortest recipe I've been able to get is here. Eval

@jimporter that recipe has an error, I think you need to quote jsonrpc-process-connection. Another nit is that you called the function eglot-minimal, but it has little to do with Eglot, it seems 😆 (I've now edited the recipe to add the quote).

jimporter · 2021-04-13T18:16:20Z

@joaotavora Hmm, it worked for me without the quote, but I'm not an expert on when quoting is required in Elisp. It seems to work (or at least, to reproduce this bug) with the quote too though.

When I submit all this to the Emacs bug tracker, I'll be sure to give the function a slightly more-generic name though. :)

joaotavora · 2021-04-13T18:17:51Z

Hmm, it worked for me without the quote, but I'm not an expert on when quoting is required in Elisp.

That's very odd, make-instance is a function and passing it an unquoted symbol means that symbol will be evaluated as a variable. And there's no variable called jsonrpc-process-connection. Please make sure.

jimporter · 2021-04-13T18:46:14Z

It looks like there's a obsoleted feature in Elisp that lets a symbol-evaluated-as-a-variable refer to the function with that name. C-h v jsonrpc-process-connection RET gives:

jsonrpc-process-connection’s value is ‘jsonrpc-process-connection’

  This variable is obsolete since 25.1;
  use \='jsonrpc-process-connection instead

So adding the quote is correct (not that I doubted it in the first place :) ), and I was inadvertently relying on an obsolete quirk of Elisp.

joaotavora · 2021-04-13T18:49:57Z

Oh, curious. Thanks for explaining. By the way, the error message is misleading. I added jsonrpc.el in Emacs 27.1 so that's a quirk of a (slightly broken) defclass. Anyway, mystery solved.

albinus · 2021-04-14T14:09:44Z

@jimporter I'm able to reproduce the problem. It is in Finsert_file_contents of src/fileio.c, you'll see the error message there. Needs further debugging (I've spent some hours on Lisp level debugging today, but I still have no idea what's going wrong).

Ergus · 2021-04-14T17:11:22Z

@jimporter I'm able to reproduce the problem. It is in Finsert_file_contents of src/fileio.c, you'll see the error message there. Needs further debugging (I've spent some hours on Lisp level debugging today, but I still have no idea what's going wrong).

Hi @albinus

Maybe you already got all this. But just in case I gave some other details here:

#667 (comment)

ZV == PT in my case.

jimporter · 2021-04-15T05:06:56Z

@albinus Thanks for taking a look. If you'd like me to file an Emacs bug to track this too, just let me know.

albinus · 2021-04-15T17:53:43Z

Hi @albinus

Maybe you already got all this. But just in case I gave some other details here:

#667 (comment)

ZV == PT in my case.

Hi @Ergus, @jimporter

Sure, that's the error case in Finsert_file_contents. I've debugged it, and it seems to be caused by a coding system problem, when inserting the stderr output into the respective buffer. I don't understand the reason (honestly, all coding system voodoo in Emacs is still magic for me). Well, stderr doesn't need coding system handling, doesn't it?

The following patch works for me, and it shows the complete stderr output in buffer *EGLOT (amhello/c++-mode) events*. It simply ignores errors from Finsert_file_contents. Could you, pls test?

*** /tmp/ediffpxpc2E	2021-04-15 19:47:36.459689919 +0200
--- /net/ford/albinus/src/tramp/lisp/tramp-sh.el	2021-04-15 19:47:33.433104914 +0200
***************
*** 2923,2937 ****
  			;; until the process is deleted.
  			(when (bufferp stderr)
  			  (with-current-buffer stderr
! 			    (insert-file-contents-literally remote-tmpstderr))
  			  ;; Delete tmpstderr file.
  			  (add-function
  			   :after (process-sentinel p)
  			   (lambda (_proc _msg)
  			     (when (file-exists-p remote-tmpstderr)
  			       (with-current-buffer stderr
! 				 (insert-file-contents-literally
! 				  remote-tmpstderr nil nil nil 'replace))
  			       (delete-file remote-tmpstderr)))))
  			;; Return process.
  			p)))
--- 2923,2939 ----
  			;; until the process is deleted.
  			(when (bufferp stderr)
  			  (with-current-buffer stderr
! 			    (ignore-errors
! 			      (insert-file-contents-literally remote-tmpstderr)))
  			  ;; Delete tmpstderr file.
  			  (add-function
  			   :after (process-sentinel p)
  			   (lambda (_proc _msg)
  			     (when (file-exists-p remote-tmpstderr)
  			       (with-current-buffer stderr
! 				 (ignore-errors
! 				   (insert-file-contents-literally
! 				    remote-tmpstderr nil nil nil 'replace)))
  			       (delete-file remote-tmpstderr)))))
  			;; Return process.
  			p)))

jimporter · 2021-04-15T21:28:09Z

@albinus That patch works for me. Thanks for looking into this!

I don't know if it will help, but after some more investigation locally, I think I'm seeing the error from Finsert_file_contents in tramp.el line 3740 in tramp-handle-insert-file-contents:

                    ;; We must ensure that `file-coding-system-alist'
                    ;; matches `local-copy'.
                    (let ((file-coding-system-alist
                           (tramp-find-file-name-coding-system-alist
                            filename local-copy)))
		      (setq result
			    (insert-file-contents    ; HERE
			     local-copy visit beg end replace))))

As you say, this might be a problem with the coding system, since the code here manipulates the coding system for the file. If I'm understanding things correctly, this code is called by the code in your patch above. Perhaps a change here would fix things; when stepping through this code, I saw that the coding system for the file (named something like tramp.XYZ.ABC) is undecided, which might be the source of the issues. Like you, the coding system in Emacs is all magic to me, so I wasn't able to get much farther than this.

Still, your patch above works for me, so maybe the rest of this comment isn't necessary. :) I'm not very familiar with what's happening here, so simply ignoring the errors might be the right thing to do.

albinus · 2021-04-16T08:05:40Z

@albinus That patch works for me. Thanks for looking into this!

I don't know if it will help, but after some more investigation locally, I think I'm seeing the error from Finsert_file_contents in tramp.el line 3740 in tramp-handle-insert-file-contents:

That's the place I've modified again and again last two days while debugging, with no luck. But I have modified my patch as below, and this looks like it is working. Although I must confess I still don't know what I'm doing.

Could you please test?

*** /tmp/ediffSMvwBw	2021-04-16 09:59:50.629040473 +0200
--- /home/albinus/src/tramp/lisp/tramp-sh.el	2021-04-16 09:53:49.174084158 +0200
***************
*** 2923,2937 ****
  			;; until the process is deleted.
  			(when (bufferp stderr)
  			  (with-current-buffer stderr
! 			    (insert-file-contents-literally remote-tmpstderr))
  			  ;; Delete tmpstderr file.
  			  (add-function
  			   :after (process-sentinel p)
  			   (lambda (_proc _msg)
  			     (when (file-exists-p remote-tmpstderr)
  			       (with-current-buffer stderr
! 				 (insert-file-contents-literally
! 				  remote-tmpstderr nil nil nil 'replace))
  			       (delete-file remote-tmpstderr)))))
  			;; Return process.
  			p)))
--- 2923,2939 ----
  			;; until the process is deleted.
  			(when (bufferp stderr)
  			  (with-current-buffer stderr
! 			    (let (last-coding-system-used)
! 			      (insert-file-contents-literally remote-tmpstderr)))
  			  ;; Delete tmpstderr file.
  			  (add-function
  			   :after (process-sentinel p)
  			   (lambda (_proc _msg)
  			     (when (file-exists-p remote-tmpstderr)
  			       (with-current-buffer stderr
! 				 (let (last-coding-system-used)
! 				   (insert-file-contents-literally
! 				    remote-tmpstderr nil nil nil 'replace)))
  			       (delete-file remote-tmpstderr)))))
  			;; Return process.
  			p)))

joaotavora · 2021-04-16T09:39:08Z

@albinus, thanks for all this work. When you're satisfied with the patch, do you think you could make a TRAMP release to GNU ELPA so that I can make Eglot depend on that and thus fix this bug?

albinus · 2021-04-16T09:44:07Z

Sure, the next Tramp 2.5.0.4 release is scheduled for end of April, on GNU ELPA. There are still some other problems in the queue, and the last week prior a release I need for extensive regression tests.

jimporter · 2021-04-16T18:55:37Z

@albinus The second patch doesn't work for me.

That's the place I've modified again and again last two days while debugging, with no luck. But I have modified my patch as below, and this looks like it is working. Although I must confess I still don't know what I'm doing.

That was roughly my experience trying to fix things in tramp-handle-insert-file-contents too. You got a lot further than me though!

albinus · 2021-04-16T19:45:17Z

@albinus The second patch doesn't work for me.

Oops, you're right. In my local setup, direct async processes were still enabled, and they don't show this problem. Cleaning Tramp cache, the error happens also with my second patch. Sorry.

I propose you shall write an Emacs bug report. I will apply my first patch (using ignore-errors to Tramp). The bug report should remind us that' there's still a problem.

jimporter · 2021-04-17T05:21:19Z

@albinus I'll try to compile all the relevant information in this issue to send to the Emacs bug tracker tomorrow. Thanks again for the help!

jimporter · 2021-04-18T17:00:03Z

Filed against Emacs as https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47861

@joaotavora With a workaround for this committed to Tramp and a bug filed against Emacs for what seems to be the underlying issue, maybe this issue can be resolved? It could also be worth keeping around at least until Tramp 2.5.0.4 is published to ELPA, and then make eglot depend on Tramp 2.5.0.4 (or if that's too big a dep to pull in, at least add a note in the README to update Tramp if you're having problems).

joaotavora · 2021-04-18T20:44:11Z

It could also be worth keeping around at least until Tramp 2.5.0.4 is published to ELPA,

Yep let's do that.

(or if that's too big a dep to pull in, at least add a note in the README to update Tramp if you're having problems).

Do you have any indication that it's big? How big?

Also, can you summarize for those who didn't follow the discussion very closely what exactly the issue was and if it's fully fixed? It seems there some kind of second issue that's not so easy to address, right?

jimporter · 2021-04-18T21:05:46Z

(or if that's too big a dep to pull in, at least add a note in the README to update Tramp if you're having problems).

Do you have any indication that it's big? How big?

Checking my git checkout of Tramp, the sources are a little under 24k lines (but that includes comments, etc).

I don't have a particularly strong opinion either way. Personally, I have no issue with adding Tramp 2.5.0.4+ as an ELPA dep since I already use the ELPA version of Tramp, but I wasn't sure if other people had opinions here. I guess I'd lean softly towards adding it as a dep, since it fixes this issue and should reduce the number of duplicate issues filed in this repo.

Also, can you summarize for those who didn't follow the discussion very closely what exactly the issue was and if it's fully fixed? It seems there some kind of second issue that's not so easy to address, right?

The issue here appears to be that, at least on some platform combinations (Windows client -> Linux server; possibly others), starting a remote LSP server fails if it writes to stderr. The problem happens when jsonrpc.el copies the stderr buffer into the events buffer, which triggers an error inside Finsert_file_contents. Tramp 2.5.0.4 will contain a workaround for this that just ignores the error so we can proceed ahead. The workaround seems to be sufficient, since I can run M-x eglot and do various LSP-related things without any (noticeable) trouble.

That said, the Tramp fix is probably just a workaround and there's some deeper problem, hence the bug filed on the Emacs tracker. Whether the bug is in Tramp, jsonrpc.el, Finsert_file_contents, or somewhere else, I don't know.

joaotavora · 2021-04-18T21:17:08Z

Whether the bug is in Tramp, jsonrpc.el

If it's in jsonrpc.el and/or can be easily fixed there, that would be reasonably good news, as that is already a dependency.

As to the rest, maybe I agree that it might be useful to make it a "soft dependency", i.e. warn in the README that one might need a newer TRAMP to get it to work in some combinations.

jimporter · 2021-04-19T03:07:28Z

If it's in jsonrpc.el and/or can be easily fixed there, that would be reasonably good news, as that is already a dependency.

The code in jsonrpc.el that triggers the error is this block. Removing that should "fix" the bug, but then you lose the ability to see stderr output in the events buffer. That's useful for debugging though, so I don't think we should remove that code if we can help it. Maybe that code is just doing something wrong and we could fix it properly though.

As to the rest, maybe I agree that it might be useful to make it a "soft dependency", i.e. warn in the README that one might need a newer TRAMP to get it to work in some combinations.

Sounds good to me. I think so long as people can see a note in the README, that should be enough. It's all the same to me though, since I've been using the GNU ELPA version of Tramp for a while now.

jimporter · 2021-04-30T21:44:29Z

TRAMP v2.5.0.4 is out now and this bug is resolved for me; I was able to remove the hacks I'd added to my .emacs and things work as expected. It might be worth keeping this issue open though in order to keep track of adding a note to the README.

jimporter · 2021-05-28T04:14:44Z

Update: commit 34ce6a47 in the Tramp repo provides a proper fix for this by using a named pipe for the stderr buffer (at least that's my understanding; I hope I've got that right).

If you like, I can add a brief note recommending Tramp 2.5.0.4+ to the README; then I think this issue can be closed.

joaotavora · 2021-05-28T07:05:40Z

If you like, I can add a brief note recommending Tramp 2.5.0.4+ to the README; then I think this issue can be closed.

Thanks. Yes. Use the small section dedicated to TRAMP, maybe link to this issue, maybe very summarily describe the issue from the non-Elisper user perspective, and mention the easiest upgrade technique.

…2.5.0.4+ * README.md (TRAMP support): Add note about updating TRAMP. (Reporting bugs): Move "reporting bugs" anchor to here.

* README.md (TRAMP support): Add note about updating TRAMP. (Reporting bugs): Move "reporting bugs" anchor to here.

jimporter · 2022-01-08T18:51:55Z

Closing per my above comment, since this is fixed in TRAMP 2.5.0.4+ and documented in the Eglot README as of #700.

joaotavora added the emacs-bug Something to be solved mostly in Emacs label Apr 5, 2021

jimporter changed the title ~~Eglot usually fails to start when using TRAMP on Windows~~ Eglot usually fails to start when using TRAMP on Windows with clangd Apr 9, 2021

jimporter mentioned this issue Apr 13, 2021

Can't make clangd work with tramp [not enough info yet] #667

Closed

jimporter mentioned this issue Apr 16, 2021

Flymake doesn't start properly when using eglot over Tramp #670

Closed

joaotavora added the tentative fix label Apr 24, 2021

jimporter added a commit to jimporter/eglot that referenced this issue May 29, 2021

Per joaotavora#662, add a note to the README about updating TRAMP to …

b4b6a23

…2.5.0.4+ * README.md (TRAMP support): Add note about updating TRAMP. (Reporting bugs): Move "reporting bugs" anchor to here.

jimporter added a commit to jimporter/eglot that referenced this issue May 29, 2021

Per joaotavora#662, add a note to the README about updating TRAMP to …

7f088b0

…2.5.0.4+ * README.md (TRAMP support): Add note about updating TRAMP. (Reporting bugs): Move "reporting bugs" anchor to here.

Ergus mentioned this issue Jun 21, 2021

Update lsp-tramp-connection emacs-lsp/lsp-mode#2531

Closed

skangas pushed a commit to jimporter/eglot that referenced this issue Jan 8, 2022

Per joaotavora#662, add a note to the README about updating TRAMP to …

dc7c9a1

…2.5.0.4+ * README.md (TRAMP support): Add note about updating TRAMP. (Reporting bugs): Move "reporting bugs" anchor to here.

skangas pushed a commit that referenced this issue Jan 8, 2022

Per #662, add a note to the README about updating TRAMP to 2.5.0.4+

761901b

* README.md (TRAMP support): Add note about updating TRAMP. (Reporting bugs): Move "reporting bugs" anchor to here.

jimporter closed this as completed Jan 8, 2022

Eglot usually fails to start when using TRAMP on Windows with clangd #662

Eglot usually fails to start when using TRAMP on Windows with clangd #662

Comments

jimporter commented Apr 5, 2021 • edited Loading

joaotavora commented Apr 5, 2021

joaotavora commented Apr 5, 2021

jimporter commented Apr 5, 2021

jimporter commented Apr 9, 2021

joaotavora commented Apr 9, 2021

jimporter commented Apr 11, 2021

jimporter commented Apr 13, 2021 • edited by joaotavora Loading

jimporter commented Apr 13, 2021 • edited Loading

jimporter commented Apr 13, 2021

Ergus commented Apr 13, 2021

albinus commented Apr 13, 2021

jimporter commented Apr 13, 2021 • edited Loading

joaotavora commented Apr 13, 2021 • edited Loading

jimporter commented Apr 13, 2021

joaotavora commented Apr 13, 2021

jimporter commented Apr 13, 2021 • edited Loading

joaotavora commented Apr 13, 2021

albinus commented Apr 14, 2021

Ergus commented Apr 14, 2021

jimporter commented Apr 15, 2021

albinus commented Apr 15, 2021

jimporter commented Apr 15, 2021

albinus commented Apr 16, 2021

joaotavora commented Apr 16, 2021

albinus commented Apr 16, 2021

jimporter commented Apr 16, 2021

albinus commented Apr 16, 2021

jimporter commented Apr 17, 2021

jimporter commented Apr 18, 2021

joaotavora commented Apr 18, 2021

jimporter commented Apr 18, 2021

joaotavora commented Apr 18, 2021

jimporter commented Apr 19, 2021

jimporter commented Apr 30, 2021

jimporter commented May 28, 2021

joaotavora commented May 28, 2021

jimporter commented Jan 8, 2022

jimporter commented Apr 5, 2021 •

edited

Loading

jimporter commented Apr 13, 2021 •

edited by joaotavora

Loading

jimporter commented Apr 13, 2021 •

edited

Loading

jimporter commented Apr 13, 2021 •

edited

Loading

joaotavora commented Apr 13, 2021 •

edited

Loading

jimporter commented Apr 13, 2021 •

edited

Loading