Elisp error connecting to clojurescript shadow-cljs repl

[evanlouie]

Originally reported at clojure-emacs/cider#2967 but I've noticed that this error only occurs with doom-emacs and I cannot recreate with spacemacs.

What did you expect to happen?

When in a shadow-cljs project, I should be able to run:

• C-c C-x j m or C-c C-x j s
• select shadow
• select my build
• have both a clj and cljs repl successfully connect to to the shadow-cljs nrepl.

What actually happened?

After running C-c C-x j s and selecting shadow an errors is printed:

error in process filter: parseclj-lex--leaf-token-value: Symbol’s function definition is void: :number
error in process filter: Symbol’s function definition is void: :number

When toggle-debug-on-error is enabled, the following is dumped:

Debugger entered--Lisp error: (void-function :number)
  :number(0)
  parseclj-lex--leaf-token-value(((:token-type . :keyword) (:form . ":source-paths") (:pos . 2)))
  (cons (parseclj-lex--leaf-token-value token) stack)
  (if (member (parseclj-lex-token-type token) (list :whitespace :comment)) stack (cons (parseclj-lex--leaf-token-value token) stack))
  parseedn-reduce-leaf((((:token-type . :lbrace) (:form . "{") (:pos . 1))) ((:token-type . :keyword) (:form . ":source-paths") (:pos . 2)) ((:tag-readers (shadow/env . identity))))
  funcall(parseedn-reduce-leaf (((:token-type . :lbrace) (:form . "{") (:pos . 1))) ((:token-type . :keyword) (:form . ":source-paths") (:pos . 2)) ((:tag-readers (shadow/env . identity))))
  (setq stack (funcall reduce-leaf stack token options))
  (cond ((parseclj-lex-leaf-token-p token) (setq stack (funcall reduce-leaf stack token options))) ((parseclj-lex-closing-token-p token) (setq stack (parseclj--reduce-coll stack token reduce-branch options))) (t (setq stack (cons token stack))))
  (while (not (or (and read-one (parseclj-single-value-p stack value-p)) (eq (parseclj-lex-token-type token) :eof))) (if (and fail-fast (parseclj-lex-error-p token)) (progn (parseclj--error "Invalid token at %s: %S" (a-get token :pos) (parseclj-lex-token-form token)))) (cond ((parseclj-lex-leaf-token-p token) (setq stack (funcall reduce-leaf stack token options))) ((parseclj-lex-closing-token-p token) (setq stack (parseclj--reduce-coll stack token reduce-branch options))) (t (setq stack (cons token stack)))) (let* ((top-value (parseclj--take-value stack value-p)) (opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)) new-stack) (while (and top-value opening-token) (setq new-stack (nthcdr (+ (length top-value) (length opening-token)) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value) options)) (setq top-value (parseclj--take-value stack value-p)) (setq opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)))) (let* ((top-value-1 (parseclj--take-value stack value-p)) (top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (opening-token (parseclj--take-token (nthcdr (+ (length top-value-1) (length top-value-2)) stack) value-p parseclj-lex--prefix-2-tokens)) new-stack) (while (and top-value-1 top-value-2 opening-token) (setq new-stack (nthcdr (apply #'+ (mapcar #'length (list top-value-1 top-value-2 opening-token))) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value-2 top-value-1) options)) (setq top-value-1 (parseclj--take-value stack value-p)) (setq top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (setq opening-token (parseclj--take-token (nthcdr (+ (length top-value-1) (length top-value-2)) stack) value-p parseclj-lex--prefix-2-tokens)))) (setq token (parseclj-lex-next)))
  (let ((fail-fast (a-get options :fail-fast t)) (read-one (a-get options :read-one)) (value-p (a-get options :value-p #'(lambda (e) (not (parseclj-lex-token-p e))))) (stack nil) (token (parseclj-lex-next))) (while (not (or (and read-one (parseclj-single-value-p stack value-p)) (eq (parseclj-lex-token-type token) :eof))) (if (and fail-fast (parseclj-lex-error-p token)) (progn (parseclj--error "Invalid token at %s: %S" (a-get token :pos) (parseclj-lex-token-form token)))) (cond ((parseclj-lex-leaf-token-p token) (setq stack (funcall reduce-leaf stack token options))) ((parseclj-lex-closing-token-p token) (setq stack (parseclj--reduce-coll stack token reduce-branch options))) (t (setq stack (cons token stack)))) (let* ((top-value (parseclj--take-value stack value-p)) (opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)) new-stack) (while (and top-value opening-token) (setq new-stack (nthcdr (+ (length top-value) (length opening-token)) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value) options)) (setq top-value (parseclj--take-value stack value-p)) (setq opening-token (parseclj--take-token (nthcdr (length top-value) stack) value-p parseclj-lex--prefix-tokens)))) (let* ((top-value-1 (parseclj--take-value stack value-p)) (top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (opening-token (parseclj--take-token (nthcdr (+ ... ...) stack) value-p parseclj-lex--prefix-2-tokens)) new-stack) (while (and top-value-1 top-value-2 opening-token) (setq new-stack (nthcdr (apply #'+ (mapcar ... ...)) stack)) (setq stack (funcall reduce-branch new-stack (car opening-token) (append (cdr opening-token) top-value-2 top-value-1) options)) (setq top-value-1 (parseclj--take-value stack value-p)) (setq top-value-2 (parseclj--take-value (nthcdr (length top-value-1) stack) value-p)) (setq opening-token (parseclj--take-token (nthcdr (+ ... ...) stack) value-p parseclj-lex--prefix-2-tokens)))) (setq token (parseclj-lex-next))) (if fail-fast (progn (let* ((token (and t (seq-find ... stack)))) (if token (parseclj--error "At position %s, unmatched %S" (a-get token :pos) (parseclj-lex-token-type token)) nil)))) (if read-one (car (parseclj--take-value stack value-p)) (car (funcall reduce-branch nil (parseclj-lex-token :root "" 1) (reverse stack) options))))
  parseclj-parser(parseedn-reduce-leaf parseedn-reduce-branch ((:tag-readers (shadow/env . identity))))
  parseedn-read(((shadow/env . identity)))
  (car (parseedn-read '((shadow/env . identity))))
  (let ((hash (car (parseedn-read '((shadow/env . identity)))))) (cider--shadow-parse-builds hash))
  (progn (insert-file-contents shadow-edn) (let ((hash (car (parseedn-read '(...))))) (cider--shadow-parse-builds hash)))
  (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash (car (parseedn-read '...)))) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash (car (parseedn-read ...)))) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash (car ...))) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
  (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ((hash ...)) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))
  (if (file-exists-p shadow-edn) (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let (...) (cider--shadow-parse-builds hash))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))))
  (let ((shadow-edn (concat (clojure-project-dir) "shadow-cljs.edn"))) (if (file-exists-p shadow-edn) (progn (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (insert-file-contents shadow-edn) (let ... ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))))))
  cider--shadow-get-builds()
  (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds))
  (or cider-shadow-default-options (car cider-shadow-watched-builds) (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds)))
  (cider-normalize-cljs-init-options (or cider-shadow-default-options (car cider-shadow-watched-builds) (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds))))
  (let* ((shadow-require "(require '[shadow.cljs.devtools.api :as shadow])") (default-build (cider-normalize-cljs-init-options (or cider-shadow-default-options (car cider-shadow-watched-builds) (completing-read "Select shadow-cljs build: " (cider--shadow-get-builds))))) (watched-builds (or (mapcar #'cider-normalize-cljs-init-options cider-shadow-watched-builds) (list default-build))) (watched-builds-form (mapconcat #'(lambda (build) (format "(shadow/watch %s)" build)) watched-builds " ")) (user-build-form "(do %s %s (shadow/nrepl-select %s))") (default-build-form "(do %s (shadow/%s))")) (if (member default-build '(":browser-repl" ":node-repl")) (format default-build-form shadow-require (string-remove-prefix ":" default-build)) (format user-build-form shadow-require watched-builds-form default-build)))
  cider-shadow-cljs-init-form()
  funcall(cider-shadow-cljs-init-form)
  (if (symbolp repl-form) (funcall repl-form) repl-form)
  (if repl-form (if (symbolp repl-form) (funcall repl-form) repl-form) (user-error "No ClojureScript REPL type %s found.  Please make ..." repl-type))
  (let* ((repl-form (and t (car (cdr (seq-find #'... cider-cljs-repl-types)))))) (if repl-form (if (symbolp repl-form) (funcall repl-form) repl-form) (user-error "No ClojureScript REPL type %s found.  Please make ..." repl-type)))
  cider-cljs-repl-form(shadow)
  (let* ((cljs-type (plist-get params :cljs-repl-type)) (repl-init-form (cider-cljs-repl-form cljs-type))) (plist-put (plist-put params :repl-init-function #'(lambda nil (cider--check-cljs cljs-type) (set (make-local-variable 'cider-cljs-repl-type) cljs-type) (cider-nrepl-send-request (list "op" "eval" "ns" (cider-current-ns) "code" repl-init-form) (cider-repl-handler (current-buffer))) (if (and (buffer-live-p nrepl-server-buffer) cider-offer-to-open-cljs-app-in-browser) (progn (cider--offer-to-open-app-in-browser nrepl-server-buffer))))) :repl-init-form repl-init-form))
  (save-current-buffer (set-buffer (or (plist-get params :--context-buffer) (current-buffer))) (let* ((cljs-type (plist-get params :cljs-repl-type)) (repl-init-form (cider-cljs-repl-form cljs-type))) (plist-put (plist-put params :repl-init-function #'(lambda nil (cider--check-cljs cljs-type) (set (make-local-variable ...) cljs-type) (cider-nrepl-send-request (list "op" "eval" "ns" ... "code" repl-init-form) (cider-repl-handler ...)) (if (and ... cider-offer-to-open-cljs-app-in-browser) (progn ...)))) :repl-init-form repl-init-form)))
  cider--update-cljs-init-function((:project-dir "~/workspace/debugging-errror/" :host #("localhost" 0 9 (ivy-index 0 idx 0)) :port 53021 :cljs-repl-type shadow))
  (plist-put (cider--update-cljs-init-function (cider--update-cljs-type (cider--check-existing-session (cider--update-host-port (cider--update-project-dir params))))) :session-name nil)
  (plist-put (plist-put (cider--update-cljs-init-function (cider--update-cljs-type (cider--check-existing-session (cider--update-host-port (cider--update-project-dir params))))) :session-name nil) :repl-type 'pending-cljs)
  (cider-nrepl-connect (plist-put (plist-put (cider--update-cljs-init-function (cider--update-cljs-type (cider--check-existing-session (cider--update-host-port (cider--update-project-dir params))))) :session-name nil) :repl-type 'pending-cljs))
  cider-connect-cljs(nil)
  funcall-interactively(cider-connect-cljs nil)
  call-interactively(cider-connect-cljs nil nil)
  command-execute(cider-connect-cljs)

Additional details:

• Config is completely fresh on whats based on develop with the only change being clojure is uncommented in init.el
• The clj repl connects successfully but the cljs connection fails with the error above.
• A conversation in Error connecting to clojurescript shadow-cljs repl clojure-emacs/cider#2967 pointed out that it seems to be an error with cl-case.
• Error seems to be related to doom. Error does not recreate on Spacemacs.
• I believe this started around mid last week (around 2021-01-13)
  • Bug doesn't happen on ded4532

Steps to reproduce:

Project Setup:

• Using latest doom-emacs
• Use default configs with only change being clojure is uncommented in init.el
• Clone any shadow-cljs template:
  • https://github.com/reagent-project/reagent-template
  • https://github.com/reagent-project/reagent-frontend-template
  • https://github.com/shadow-cljs/lein-template

Run it with either of:

• via C-c C-x C-c s:
  • Start a shadow-cljs repl: npx shadow-cljs watch app
  • in emacs: C-c C-x C-c s, select localhost, select the shown port, select shadow
• via C-c C-x j s or C-c C-x j m
  • Run C-c C-x j s or C-c C-x j m in the project
  • Select shadow

System information:

Mac:

• MacOS: 11.1
• Emacs: 27.1 (Emacs Mac Port)

Linux:

• Pop!_OS 20.04
• Emacs: 27.1 (from ppa:kelleyk/emacs)

[evanlouie]

After some testing, i have found the offending commit: cfb8a86
However, it appears one of the commits by @hlissner made today (2020-01-23) between cb09f17 and 4aed841 has resolved it!

[wojtyniak]

I'm having the same issue and upgrading doom doesn't help. @evanlouie, have you changed anything in your setup by any chance?

EDIT: running doom build fixed the issue! :)

[hlissner]

TL;DR This is a byte-code issue. Recompiling the responsible package (parseclj) should fix it (run doom build).

Why it happens: A needed macro (presumably cl-case) isn't defined when the parseclj-lex.el module is compiled, so calls to this macro are left unexpanded in the resulting bytecode. This means its containing forms (e.g. (:number ...)) are treated as regular function calls, except no function named :number is defined. Cue your error.

The reason this works in Spacemacs: because it loads cl-lib in the same session that it installs and byte-compiles its packages, so that macro is always available. Straight (our package manager), as of a couple weeks ago, byte-compiles packages in an isolated Emacs child process, where nothing but the package (and the packages it directly loads) are loaded (and cl-macs is presumably missing).

Hopefully, this PR will prevent issues like these in the future.