Add customizable terminal backends, support for vterm

[tpapp]

Factor out code interacting with the terminal using generic functions.

Add backend code for emacs-libvterm.

Disables byte compilation do get conditional code to work.

[antoine-levitt]

Interesting! What's your experience with it?

[tpapp]

So far, smooth, but I am still testing it.

[ghost]

Thanks for setting this up. I haven't figured out why, but I'm now unable to switch my term mode (to copy/yank) with the default term. I've changed a lot in my config since the last time I've tried, including starting to use Doom and EXWM.

I followed the README for setting the term backend, which in Doom looks like:

(after! julia-repl
  (setq julia-repl-terminal-backend (make-julia-repl--buffer-vterm)))

Unfortunately, this method, and several others I've tried, all fail with:

Symbol’s function definition is void: make-julia-repl--buffer-vterm

For now, I just edited julia-repl.el to set the term backend directly (which works):

(defvar julia-repl-terminal-backend
  (make-julia-repl--buffer-vterm)
  "Terminal backend. FIXME Currently experimental, only modify if you know what you are doing.")

I confirmed that vterm's copy-mode works, so I can use julia-repl again.

[tpapp]

@mindlike: I suspect it is a load order issue, but I am not familiar with Doom.

[tpapp]

For those following this PR: the remaining outstanding issue is that clicking on source locations in error messages can kill the Emacs process. I am working on debugging this.

[ghost]

FWIW, the only way I could get this to work in Doom was with an explicit require.

(use-package! julia-repl
  :config
  (require 'vterm)
  (setq julia-repl-terminal-backend (make-julia-repl--buffer-vterm)))

[ghost]

For those following this PR: the remaining outstanding issue is that clicking on source locations in error messages can kill the Emacs process. I am working on debugging this.

I've been able to click source locations emitted by Debugger without issue so far.

[tpapp]

@mindlike: clickable locations recently work for me too, possibly because of an update of libvterm.

Doom appears to do a lot of transformations to make startup faster, which I suspect are the issue. The only thing I could find is

https://github.com/hlissner/doom-emacs/blob/develop/docs/faq.org#void-variable-and-void-function-errors-on-startup

I am happy to fix anything in this package once you identify the cause, but I can't help you with debugging Doom-specific issues. Maybe you could ask on the Doom forum.

[christopher-dG]

I just tried this out and got an error when running (julia-repl):

Debugger entered--Lisp error: (cl-no-applicable-method julia-repl--locate-live-buffer #s(julia-repl--buffer-vterm) "julia")
  signal(cl-no-applicable-method (julia-repl--locate-live-buffer #s(julia-repl--buffer-vterm) "julia"))
  cl-no-applicable-method(#s(cl--generic :name julia-repl--locate-live-buffer :dispatches ((1 #s(cl--generic-generalizer :name cl--generic-t-generalizer :priority 0 :tagcode-function #f(compiled-function (name &rest _) #<bytecode 0x1ffd13471bf7>) :specializers-function #f(compiled-function (tag &rest _) #<bytecode 0x1ffd13471be7>))) (0 #s(cl--generic-generalizer :name cl--generic-struct-generalizer :priority 50 :tagcode-function cl--generic-struct-tag :specializers-function cl--generic-struct-specializers) #s(cl--generic-generalizer :name cl--generic-t-generalizer :priority 0 :tagcode-function #f(compiled-function (name &rest _) #<bytecode 0x1ffd13471bf7>) :specializers-function #f(compiled-function (tag &rest _) #<bytecode 0x1ffd13471be7>)))) :method-table (#s(cl--generic-method :specializers (julia-repl--buffer-ansi-term t) :qualifiers nil :uses-cnm nil :function (closure (cl-struct-julia-repl--buffer-ansi-term-tags t) (_terminal-backend name) (progn (let* (...) (if inferior-buffer ... ...)))))) :options nil) #s(julia-repl--buffer-vterm) "julia")
  apply(cl-no-applicable-method #s(cl--generic :name julia-repl--locate-live-buffer :dispatches ((1 #s(cl--generic-generalizer :name cl--generic-t-generalizer :priority 0 :tagcode-function #f(compiled-function (name &rest _) #<bytecode 0x1ffd13471bf7>) :specializers-function #f(compiled-function (tag &rest _) #<bytecode 0x1ffd13471be7>))) (0 #s(cl--generic-generalizer :name cl--generic-struct-generalizer :priority 50 :tagcode-function cl--generic-struct-tag :specializers-function cl--generic-struct-specializers) #s(cl--generic-generalizer :name cl--generic-t-generalizer :priority 0 :tagcode-function #f(compiled-function (name &rest _) #<bytecode 0x1ffd13471bf7>) :specializers-function #f(compiled-function (tag &rest _) #<bytecode 0x1ffd13471be7>)))) :method-table (#s(cl--generic-method :specializers (julia-repl--buffer-ansi-term t) :qualifiers nil :uses-cnm nil :function (closure (cl-struct-julia-repl--buffer-ansi-term-tags t) (_terminal-backend name) (progn (let* (...) (if inferior-buffer ... ...)))))) :options nil) (#s(julia-repl--buffer-vterm) "julia"))
  #f(compiled-function (&rest args) #<bytecode 0x156bd80c6c2d>)(#s(julia-repl--buffer-vterm) "julia")
  apply(#f(compiled-function (&rest args) #<bytecode 0x156bd80c6c2d>) #s(julia-repl--buffer-vterm) "julia")
  julia-repl--locate-live-buffer(#s(julia-repl--buffer-vterm) "julia")
  (let* ((name (julia-repl--inferior-buffer-name executable-key suffix)) (live-buffer (julia-repl--locate-live-buffer terminal-backend name))) (if live-buffer live-buffer (let ((executable-record (julia-repl--executable-record executable-key)) (switches julia-repl-switches)) (julia-repl--complete-executable-record! executable-record) (let* ((executable-path (car (cdr executable-record))) (basedir (plist-get (cdr ...) :basedir)) (inferior-buffer (julia-repl--make-buffer terminal-backend name executable-path (if switches ...)))) (if julia-repl-compilation-mode (progn (julia-repl--setup-compilation-mode inferior-buffer basedir))) (julia-repl--run-hooks inferior-buffer) (let* ((v inferior-buffer) (v suffix)) (save-current-buffer (set-buffer v) (set (make-local-variable ...) v))) inferior-buffer))))
  (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '(:executable-key :suffix :terminal-backend :allow-other-keys)) (setq --cl-keys-- (cdr (cdr --cl-keys--)))) ((car (cdr (memq ... --cl-rest--))) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:executable-key :s..." (car --cl-keys--)))))) (let* ((name (julia-repl--inferior-buffer-name executable-key suffix)) (live-buffer (julia-repl--locate-live-buffer terminal-backend name))) (if live-buffer live-buffer (let ((executable-record (julia-repl--executable-record executable-key)) (switches julia-repl-switches)) (julia-repl--complete-executable-record! executable-record) (let* ((executable-path (car ...)) (basedir (plist-get ... :basedir)) (inferior-buffer (julia-repl--make-buffer terminal-backend name executable-path ...))) (if julia-repl-compilation-mode (progn (julia-repl--setup-compilation-mode inferior-buffer basedir))) (julia-repl--run-hooks inferior-buffer) (let* ((v inferior-buffer) (v suffix)) (save-current-buffer (set-buffer v) (set ... v))) inferior-buffer)))))
  (let* ((executable-key (car (cdr (or (plist-member --cl-rest-- ':executable-key) (list nil (julia-repl--get-executable-key)))))) (suffix (car (cdr (or (plist-member --cl-rest-- ':suffix) (list nil julia-repl-inferior-buffer-name-suffix))))) (terminal-backend (car (cdr (or (plist-member --cl-rest-- ':terminal-backend) (list nil julia-repl-terminal-backend)))))) (progn (let ((--cl-keys-- --cl-rest--)) (while --cl-keys-- (cond ((memq (car --cl-keys--) '...) (setq --cl-keys-- (cdr ...))) ((car (cdr ...)) (setq --cl-keys-- nil)) (t (error "Keyword argument %s not one of (:executable-key :s..." (car --cl-keys--)))))) (let* ((name (julia-repl--inferior-buffer-name executable-key suffix)) (live-buffer (julia-repl--locate-live-buffer terminal-backend name))) (if live-buffer live-buffer (let ((executable-record (julia-repl--executable-record executable-key)) (switches julia-repl-switches)) (julia-repl--complete-executable-record! executable-record) (let* ((executable-path ...) (basedir ...) (inferior-buffer ...)) (if julia-repl-compilation-mode (progn ...)) (julia-repl--run-hooks inferior-buffer) (let* (... ...) (save-current-buffer ... ...)) inferior-buffer))))))
  julia-repl-inferior-buffer()
  (switch-to-buffer-other-window (julia-repl-inferior-buffer))
  julia-repl()
  eval((julia-repl) t)
  eval-expression((julia-repl) nil nil 127)
  funcall-interactively(eval-expression (julia-repl) nil nil 127)
  call-interactively(eval-expression nil nil)
  command-execute(eval-expression)

I have this PR branch checked out and the following config:

(require 'julia-repl)
(add-hook 'julia-mode-hook 'julia-repl-mode)
(setq julia-repl-terminal-backend (make-julia-repl--buffer-vterm))

[christopher-dG]

The above is likely a false alarm, I just tried it again after figuring out use-package for local paths and it worked:

(use-package julia-repl
  :load-path "lisp/julia-repl"
  :hook (julia-mode . julia-repl-mode)
  :config (setq julia-repl-terminal-backend (make-julia-repl--buffer-vterm))
  :bind (:map julia-repl-mode-map
              ("C-c C-j" . julia-repl)))

[tpapp]

To anyone experimenting with this branch: the backend selection API changed to

(julia-repl-set-terminal-backend 'vterm)

no longer exposing the internal structures. I am hoping that this will fix the load time issue noticed by @christopher-dG.

In the meantime, I think that the remaining bugs in vterm are being fixed. I will test the latest version a bit more then merge this.

[tpapp]

A quick update to anyone following this: I am not yet merging this because of akermu/emacs-libvterm#316 which causes a segfault if someone clicks on a source location. Otherwise the PR is ready though.

I understand that several users would like to use vterm as a backend, since it is really nice. Until that issue is fixed, I could introduce a workaround that either

1. turns of compilation-shell-minor-mode, no clicking on source locations with vterm then.
2. since I don't really need pwd for this to work, introduce a workarond for that feature, to be removed later

[tpapp]

Update: this branch now has a workaround for the libvterm problem (until that is fixed). I will experiment with it and merge when I found it stable.

[mauro3]

There is now competition https://github.com/shg/julia-vterm.el.

[mauro3]

Also, for me this PR seems to work well.

[tpapp]

Glad to hear this (and the competition too). I am close to merging, but vterm and ansi-term process options to the executable differently so that's something I still need to fix in this PR.

[martenlienen]

This should be

(let ((vterm-shell (concat executable-path " " (mapconcat #'identity switches " ")))

At the moment, all options are concatenated without spaces like -Jsysimage.so--threads8.

[tpapp]

Thanks! I fixed this with s-join since we are using that package anyway.

[martenlienen]

I am just setting this up and everything is nice and speedy. I had one small hiccup with the switches as I commented in the code. Another thing I noticed but could not really fix is that if you have (add-hook 'julia-repl-hook #'julia-repl-use-emacsclient) sending ENV["JULIA_EDITOR"] = "emacsclient"; right after initializing the inferior shell, somehow julia's paste mode handling seems to not be set up yet and the opening bracket triggers julia's bracket completion. In the end, the shell runs ENV["JULIA_EDITOR"] = "emacsclient";] which fails.

I am looking forward to seeing this merged. Also makes it easier to contribute :)

[martenlienen]

I just realized that julia does not have bracket autocompletion out of the box. So the bracket issue above is an interaction with OhMyREPL and should not be too concerning.

[antoine-levitt]

Hi, any plans to merge this soon? Looks like @cqql 's comment above resolves the last issue you were talking about @tpapp ?

[martenlienen]

The interaction with OhMyREPL can be avoided with the alternative setup for JULIA_EDITOR in #100.

[antoine-levitt]

Thanks!

[antoine-levitt]

Mind tagging an ELPA release with this? Makes it simpler to use.

[tpapp]

@antoine-levitt: I assuming you mean MELPA Stable? I added a tag.

[antoine-levitt]

I think that's what I meant, yes. Thanks!