Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZMQ subprocess error #297

Open
e-eight opened this issue Oct 25, 2020 · 21 comments
Open

ZMQ subprocess error #297

e-eight opened this issue Oct 25, 2020 · 21 comments

Comments

@e-eight
Copy link

e-eight commented Oct 25, 2020

I recently re-installed emacs-jupyter, and now whenever I am trying to run a jupyter-python source block in Org-mode I am getting the following error:

error in process filter: zmq--subprocess-filter: Error in ZMQ subprocess: void-variable, (jupyter-channel)
error in process filter: Error in ZMQ subprocess: void-variable, (jupyter-channel)
cl--assertion-failed: Assertion failed: (process-live-p process)

This is with natively compiled emacs 28. With emacs 27, not natively compiled, it appears to be working properly.

@smsegal
Copy link

smsegal commented Oct 30, 2020

I'm experiencing the same problem with native-comp on emacs28.
If anyone is experiencing the same error and using straight.el, you can disable native compilation for the jupyter package by specifying :no-native-compile t in your straight recipe. E.g

(use-package jupyter
  :straight (:no-native-compile t))

@mbarton98
Copy link

I was able to get this to work since I'm not using straight.
(setq comp-deferred-compilation-deny-list (list "jupyter"))

@veprbl
Copy link

veprbl commented Mar 22, 2021

Should be fixed by #248

@Jonghyun-Yun
Copy link

I am unable to use emacs-jupyter in Emacs 28.0.91 with native-comp. Evaluating a jupyter-python block leads to the error below. I am using Doom Emacs.

Check for compatible module binary to download? (y or n) y
Downloading https://github.com/nnicandro/emacs-zmq/releases/download/v0.10.10/emacs-zmq-x86_64-apple-darwin17.4.0.tar.gz
Verifying sha256 signature of emacs-zmq-x86_64-apple-darwin17.4.0.tar.gz
Wrote /Users/yunj/doom-emacs/.local/straight/build-28.0.91/zmq/emacs-zmq-x86_64-apple-darwin17.4.0.tar.gz
Jupyter: (cl-assertion-failed (not enable-multibyte-characters))
org-babel-execute-src-block: No org-babel-execute function for jupyter-python!

@Panadestein
Copy link

Pardon the necrobumping. I think this bug is alive again:

Executing Jupyter-Python code block at position 114...
Starting python3 kernel process...done
Warning: Unknown type: zmq-socket
error in process filter: zmq--subprocess-filter: Error in ZMQ subprocess: (void-variable )
error in process filter: Error in ZMQ subprocess: (void-variable )
Warning: Unknown type: zmq-socket
Assertion failed: (process-live-p process)
finalizer failed: (cl-no-primary-method jupyter-kill-kernel #s(jupyter-kernel-process-manager (#<finalizer used>) jupyter--kernel-managers #s(jupyter-spec-kernel (#<finalizer> #<finalizer used>) ("python3" "/nix/store/pwr103hd66pfl4il84ldmljxc055wn9w-python3-3.10.9-env/share/jupyter/kernels/python3" :argv ["python" "-m" "ipykernel_launcher" "-f" "{connection_file}"] :env nil :display_name "Python 3 (ipykernel)" :language "python" :interrupt_mode "signal" :metadata (:debugger t)) #s(jupyter-session (:kernel_name "python3" :transport "tcp" :ip "127.0.0.1" :signature_scheme "hmac-sha256" :key "42fc60d6-c899-4771-b5cb-fc4c5082b4ae" :hb_port 33749 :stdin_port 45959 :control_port 40559 :shell_port 41345 :iopub_port 45179) "fa4e6e14-2019-4f8c-8fbb-53f1fcf94184" "42fc60d6-c899-4771-b5cb-fc4c5082b4ae") #<process jupyter-kernel-python3<1>>) #s(jupyter-zmq-channel :control #s(jupyter-session (:kernel_name "python3" :transport "tcp" :ip "127.0.0.1" :signature_scheme "hmac-sha256" :key "42fc60d6-c899-4771-b5cb-fc4c5082b4ae" :hb_port 33749 :stdin_port 45959 :control_port 40559 :shell_port 41345 :iopub_port 45179) "fa4e6e14-2019-4f8c-8fbb-53f1fcf94184" "42fc60d6-c899-4771-b5cb-fc4c5082b4ae") "tcp://127.0.0.1:40559" #<user-ptr ptr=0x9cda3e0 finalizer=0x7fa620e7fa10>)))

Full backtrace:

Debugger entered--Lisp error: (zmq-subprocess-error (void-variable ))
  signal(zmq-subprocess-error ((void-variable )))
  zmq--subprocess-filter(#<process zmq> "\nError: void-variable ()\n  debug-early-backtrace(...")
  accept-process-output(nil 0.25)
  jupyter-ioloop-wait-until(#<jupyter-zmq-channel-ioloop jupyter-zmq-channel-ioloop-27c1756> start identity)
  #f(compiled-function (arg1 arg2 &rest rest) #<bytecode 0x186427b2e0e2e74c>)(#<jupyter-zmq-channel-ioloop jupyter-zmq-channel-ioloop-27c1756> #f(compiled-function (event) #<bytecode 0x70a192d7e5ae66e>))
  apply(#f(compiled-function (arg1 arg2 &rest rest) #<bytecode 0x186427b2e0e2e74c>) #<jupyter-zmq-channel-ioloop jupyter-zmq-channel-ioloop-27c1756> #f(compiled-function (event) #<bytecode 0x70a192d7e5ae66e>))
  jupyter-ioloop-start(#<jupyter-zmq-channel-ioloop jupyter-zmq-channel-ioloop-27c1756> #f(compiled-function (event) #<bytecode 0x70a192d7e5ae66e>))
  #f(compiled-function (comm) #<bytecode -0xb25cd1cc32964dd>)(#<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee>)
  apply(#f(compiled-function (comm) #<bytecode -0xb25cd1cc32964dd>) #<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee> nil)
  jupyter-comm-start(#<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee>)
  #f(compiled-function (comm obj) #<bytecode 0x9adde364f45440a>)(#<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee> #<jupyter-org-client jupyter-org-client-229da26>)
  apply(#f(compiled-function (comm obj) #<bytecode 0x9adde364f45440a>) #<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee> #<jupyter-org-client jupyter-org-client-229da26>)
  #f(compiled-function (arg &rest args) #<bytecode -0xdee67ab66deb581>)(#<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee> #<jupyter-org-client jupyter-org-client-229da26>)
  apply(#f(compiled-function (arg &rest args) #<bytecode -0xdee67ab66deb581>) #<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee> #<jupyter-org-client jupyter-org-client-229da26> nil)
  jupyter-comm-add-handler(#<jupyter-channel-ioloop-comm jupyter-channel-ioloop-comm-229daee> #<jupyter-org-client jupyter-org-client-229da26>)
  #f(compiled-function (client) #<bytecode -0x1b75d991ee90683c>)(#<jupyter-org-client jupyter-org-client-229da26>)
  apply(#f(compiled-function (client) #<bytecode -0x1b75d991ee90683c>) #<jupyter-org-client jupyter-org-client-229da26> nil)
  jupyter-start-channels(#<jupyter-org-client jupyter-org-client-229da26>)
  jupyter-start-new-kernel("python3" jupyter-org-client)
  jupyter-run-repl("python3" nil nil jupyter-org-client)
  #f(compiled-function (session kernel) #<bytecode -0x43deee6f2e03c41>)(#s(org-babel-jupyter-session :name "cdwac") "python3")
  apply(#f(compiled-function (session kernel) #<bytecode -0x43deee6f2e03c41>) (#s(org-babel-jupyter-session :name "cdwac") "python3"))
  #f(compiled-function (&rest args) #<bytecode 0x19717a1d2f972332>)()
  #f(compiled-function (cl--cnm session kernel) #<bytecode 0xa911f1009b956f5>)(#f(compiled-function (&rest args) #<bytecode 0x19717a1d2f972332>) #s(org-babel-jupyter-session :name "cdwac") "python3")
  apply(#f(compiled-function (cl--cnm session kernel) #<bytecode 0xa911f1009b956f5>) #f(compiled-function (&rest args) #<bytecode 0x19717a1d2f972332>) (#s(org-babel-jupyter-session :name "cdwac") "python3"))
  #f(compiled-function (session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x59c5e4364f225d6>)(#s(org-babel-jupyter-session :name "cdwac") "python3")
  apply(#f(compiled-function (session kernel) "Rename the returned client's REPL buffer to include SESSION's name.\nAlso set `jupyter-include-other-output' to nil for the session so\nthat output produced by other clients do not get handled by the\nclient." #<bytecode 0x59c5e4364f225d6>) #s(org-babel-jupyter-session :name "cdwac") "python3")
  org-babel-jupyter-initiate-client(#s(org-babel-jupyter-session :name "cdwac") "python3")
  org-babel-jupyter-initiate-session-by-key("cdwac" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "cdwac")))
  org-babel-jupyter-initiate-session("cdwac" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "cdwac")))
  org-babel-execute:jupyter-python("import numpy as np\nimport matplotlib\nimport matplo..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:kernel . "python3") (:async . "no") (:session . "cdwac")))
  org-babel-execute-src-block(nil ("jupyter-python" "import numpy as np\nimport matplotlib\nimport matplo..." ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session . "cdwac") (:async . "no") (:kernel . "python3") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 114 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)
  command-execute(org-ctrl-c-ctrl-c)

@nakkaya
Copy link

nakkaya commented Jan 28, 2023

Pardon the necrobumping. I think this bug is alive again:

I can also confirm bug is live. Stopping emacs from compiling the package fixes the problem.

@havarddj
Copy link

Hi! I tried
(setq native-comp-deferred-compilation-deny-list '("jupyter*.el" "zmq*.el"))
but it doesn't seem to prevent it from compiling. What do you use? Running emacs 29 on macOS

@Panadestein
Copy link

For reference, I am using Emacs from master in NixOS, and emacs-jupyter with straight.el

@dakra
Copy link
Contributor

dakra commented Jan 30, 2023

@havarddj native-comp-deferred-compilation-deny-list is a list of regexps.
So you probably want something like "zmq.*\\.el" etc.

@nakkaya
Copy link

nakkaya commented Jan 30, 2023

@havarddj I was able to stop it from compiling using,

(add-to-list 'native-comp-deferred-compilation-deny-list ".*jupyter.*")

I don't have zmq on the list but worked for me. (Emacs 28.2 on OS X)

@havarddj
Copy link

RIght, seems like my problem is actually with zmq failing some tests. Thanks for the help!

@benneti
Copy link

benneti commented Jan 30, 2023

@Panadestein were you able to fix it, I think I run into the exact same problem (also on nixos)

@Panadestein
Copy link

Panadestein commented Jan 31, 2023

@benneti unfortunately, it is still broken for me. I have tried to prevent native compilation with straight, as suggested above:

(use-package jupyter
  :straight (:build (:not native-compile)))

But the problem persists. I still get the error:

Starting python3 kernel process...done
Warning: Unknown type: zmq-socket
error in process filter: zmq--subprocess-filter: Error in ZMQ subprocess: (void-variable )
error in process filter: Error in ZMQ subprocess: (void-variable )
Warning: Unknown type: zmq-socket

The other ways of preventing native compilation suggested in this issue do not have any effect on my config.

@benneti
Copy link

benneti commented Jan 31, 2023

I tried this too, even cleaning out the elm in the nix store (they seem to be generated even with the build recipe). But it didn't help either but I am not sure whether I got all the eln files either.

@Panadestein
Copy link

Panadestein commented Jan 31, 2023

Oh, you can go ahead and purge the build folder of jupyter, straight won't do it for you. Still, I have checked, and I don't have any .eln file, but the issue persists. How are you managing your Python environment in Nix? I am worried it could be a NixOS exclusive problem.

@benneti
Copy link

benneti commented Jan 31, 2023

I use python.withPackages (globally installed in my user profile).
On the other hand going back to emacs 28 fixes the problem, which in my opinion makes it equally likely that the problem is emacs 29.

@iskur
Copy link

iskur commented Jan 31, 2023

@Panadestein The problem is not limited to NiXOS. I also experience it on ArchLinux with the emacs-git package from AUR (version 30).

@Panadestein
Copy link

@iskur @benneti nice to know it is not exclusive to NixOS. Well, I do not know how to debug this further, maybe someone else has an idea.

@havarddj
Copy link

havarddj commented Jan 31, 2023

@iskur @benneti @Panadestein Switching from emacs 29 to 28 fixed the problem on macOS, so it's likely the same problem.

Running make test in the emacs-zmq directory (after copying zmq-tests.el from the github repo) gave the following output:

emacs -nw -Q -batch -L . -l ert -l zmq-tests.el \
		--eval "(ert-run-tests-batch-and-exit)"
Loading /Users/havard/.emacs.d/elpa/zmq-20220510.1820/emacs-zmq.dylib (module)...
ZMQ Version: 4.3.1
Running 11 tests (2023-01-29 13:25:44+0000, selector ‘t’)
   passed   1/11  zmq-close-socket-once (0.002947 sec)
   passed   2/11  zmq-contexts (0.000047 sec)
   passed   3/11  zmq-encryption (0.000021 sec)
   passed   4/11  zmq-globrefs (0.235227 sec)
   passed   5/11  zmq-messages (0.001528 sec)
   passed   6/11  zmq-polling (2.018907 sec)
Test zmq-send-unicode backtrace:
  zmq-recv(#<user-ptr ptr=0x600003ba4140 finalizer=0x10b72a784> nil)
  zmq-recv-decoded(#<user-ptr ptr=0x600003ba4140 finalizer=0x10b72a784
  (setq s (zmq-recv-decoded s2 'utf-16))
  (let ((u "çπ§") (s nil)) (zmq-send s1 u) (setq s (zmq-recv s2)) (let
  (progn (let ((u "çπ§") (s nil)) (zmq-send s1 u) (setq s (zmq-recv s2
  (unwind-protect (progn (let ((u "çπ§") (s nil)) (zmq-send s1 u) (set
  (let* ((--cl-rest-- (zmq-create-bound-pair ctx zmq-PAIR zmq-PAIR nil
  (let* ((ctx (zmq-context))) (let* ((--cl-rest-- (zmq-create-bound-pa
  (let ((ert--infos (cons (cons "Info: " "Sending unicode messages") e
  (let ((ert--infos (cons (cons "Info: " "Unicode messages") ert--info
  (closure (t) nil (let ((ert--infos (cons (cons "Info: " "Unicode mes
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name zmq-send-unicode :documentation nil :
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests ... :test-map
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  command-line-1(("-L" "." "-l" "ert" "-l" "zmq-tests.el" "--eval" "(e
  command-line()
  normal-top-level()
Test zmq-send-unicode condition:
    Info: Unicode messages
    Info: Sending unicode messages
    (wrong-type-argument utf-8-string-p "\376\377\0\0\247use")
   FAILED   7/11  zmq-send-unicode (0.002582 sec) at zmq-tests.el:185
Test zmq-sockets backtrace:
  signal(wrong-type-argument (utf-8-string-p "\376\377\0t\0\0t\0\0\0")
  apply(signal (wrong-type-argument (utf-8-string-p "\376\377\0t\0\0t\
  (setq value-332 (apply fn-330 args-331))
  (unwind-protect (setq value-332 (apply fn-330 args-331)) (setq form-
  (if (unwind-protect (setq value-332 (apply fn-330 args-331)) (setq f
  (let (form-description-334) (if (unwind-protect (setq value-332 (app
  (let ((value-332 'ert-form-evaluation-aborted-333)) (let (form-descr
  (let* ((fn-330 #'equal) (args-331 (condition-case err (let ((signal-
  (let ((topic "tést")) (zmq-socket-set-encoded sock zmq-ROUTING-ID to
  (let ((ert--infos (cons (cons "Info: " "Unicode options") ert--infos
  (let ((sock (zmq-socket ctx zmq-SUB))) (let ((ert--infos (cons (cons
  (let ((sock (zmq-socket ctx zmq-PULL))) (let ((ert--infos (cons (con
  (let ((ert--infos (cons (cons "Info: " "Socket options") ert--infos)
  (let* ((ctx (zmq-context)) (endpoint "tcp://127.0.0.1:5345") (s (zmq
  (closure (t) nil (let* ((ctx (zmq-context)) (endpoint "tcp://127.0.0
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name zmq-sockets :documentation nil :body 
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests ... :test-map
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  command-line-1(("-L" "." "-l" "ert" "-l" "zmq-tests.el" "--eval" "(e
  command-line()
  normal-top-level()
Test zmq-sockets condition:
    Info: Socket options
    Info: Unicode options
    (wrong-type-argument utf-8-string-p "\376\377\0t\0\0t\0\0\0")
   FAILED   8/11  zmq-sockets (0.002320 sec) at zmq-tests.el:208
Test zmq-subprocess backtrace:
  signal(ert-test-failed (((should-error (setq proc (zmq-start-process
  ert-fail(((should-error (setq proc (zmq-start-process (lambda (_a _b
  (if errorp401 nil (ert-fail (append (funcall form-description-fn-402
  (let ((errorp401 nil) (form-description-fn-402 #'(lambda nil form-de
  (let (form-description-400) (let ((errorp401 nil) (form-description-
  (let ((value-399 (gensym "ert-form-evaluation-aborted-"))) (let (for
  (let ((ert--infos (cons (cons "Info: " "Only functions with 0 or 1 a
  (progn (let ((value-395 (gensym "ert-form-evaluation-aborted-"))) (l
  (unwind-protect (progn (let ((value-395 (gensym "ert-form-evaluation
  (let (proc) (unwind-protect (progn (let ((value-395 (gensym "ert-for
  (let ((ert--infos (cons (cons "Info: " "Validating sexp") ert--infos
  (closure (t) nil (let ((ert--infos (cons (cons "Info: " "Validating 
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name zmq-subprocess :documentation nil :bo
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests ... :test-map
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  command-line-1(("-L" "." "-l" "ert" "-l" "zmq-tests.el" "--eval" "(e
  command-line()
  normal-top-level()
Test zmq-subprocess condition:
    Info: Validating sexp
    Info: Only functions with 0 or 1 arguments
    (ert-test-failed
     ((should-error
       (setq proc
	     (zmq-start-process ...)))
      :form
      (setq proc
	    (zmq-start-process
	     #'(lambda ...)))
      :value #<process zmq> :fail-reason "did not signal an error"))
   FAILED   9/11  zmq-subprocess (0.000619 sec) at zmq-tests.el:332
   passed  10/11  zmq-utility (0.000200 sec)
   passed  11/11  zmq-wrong-object-type (0.000041 sec)

Ran 11 tests, 8 results as expected, 3 unexpected (2023-01-29 13:25:46+0000, 2.412057 sec)

3 unexpected results:
   FAILED  zmq-send-unicode
   FAILED  zmq-sockets
   FAILED  zmq-subprocess

make: *** [test] Error 1 ```

@benneti
Copy link

benneti commented Feb 7, 2023

On latest master (i.e. bc3b78b for jupyter and 00dea2f9f88cd3e723d3b909aea26313dd33f9c6 for zmq) it is working for me using the pgtk branch (apart from some json warnings).

@hvzzzz
Copy link

hvzzzz commented Jun 21, 2023

I tried rebuilding emacs-zmq.so running make in the emacs-zmq directory created by straight, in my case .emacs.d/.local/straight/build-29.0.91/zmq, it worked for me on Emacs 29.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests