Jupyter fails to function after upgrade

[sternj]

I confirm that...

• I have searched the issue tracker, documentation, FAQ, Discourse, and Google, in case this issue has already been reported/resolved.

• I have read "How to Debug Issues", and will use it to provide as much information about this issue as possible.

• The issue can be reproduced on the latest available commit of Doom.

• The issue can be reproduced on a stable release of Emacs, such as 27 or 28. (Doom does not support development builds like 29+ or any version ending in .50 or .9x)

Expected behavior

When I have emacs-jupyer loaded with (org ... +jupyter ...) in init.el, the jupyter-python environment should be available in Babel

Current behavior

When I attempt to execute a jupyter-python block in an org file, I get the following error message:

org-babel-execute-src-block: No org-babel-execute function for jupyter-python!

The most recent messages are as follows:

Company: An error occurred in auto-begin
Company: backend company-capf error "Invalid search bound (wrong side of point)" with args (prefix)
Company: An error occurred in auto-begin
Company: backend company-capf error "Wrong type argument: integer-or-marker-p, nil" with args (prefix)
Company: An error occurred in auto-begin
Company: backend company-capf error "Wrong type argument: integer-or-marker-p, nil" with args (prefix)
Company: An error occurred in auto-begin
Company: backend company-capf error "Wrong type argument: integer-or-marker-p, nil" with args (prefix)
Quit [3 times]
mouse-minibuffer-check: Minibuffer window is not active
Quit
org-babel-execute-src-block: No org-babel-execute function for jupyter-python!
Your doom-info was copied to the clipboard.

I created this issue in the emacs-jupyter repo, which linked to (this issue)[https://github.com/org-roam/org-roam/issues/2361] in the org-roam repo, which led me to try the solution from this comment on it, which didn't work. Looking at emacs-jupyter, it seems that the canonical way to activate it is with

(org-babel-do-load-languages
 'org-babel-load-languages
 '((emacs-lisp . t)
   (julia . t)
   (python . t)
   (jupyter . t)))

and the 'org-babel-load-languages that I see upon inspection has jupyter-python in it instead, but I'm unsure if that's a problem in any way. I also see other jupyter functions working in emacs, just not this one, so the module is loading correctly. I have attempted adding the do-load-langauges hook myself in config.el but this too has failed to resolve the issue.

Steps to reproduce

1. Open an org file
2. Create a block with the header #+begin_src jupyter-python :session whatever
3. Execute with C-c C-c

System Information

https://pastebin.com/JNM6Vfct

[sternj]

It seems like the folks in the Jupyter repo may have passed the buck prematurely, reverting to a slightly older emacs-jupyter version seems to resolve the issue. Leaving it up to the maintainers here as to whether they think it's worthwhile to have a linked issue

[UndeadKernel]

Which commit are you using with emacs-jupyter?
I'm also experiencing issues with the current version.

[thiagotps]

From this issue https://github.com/emacs-jupyter/jupyter/issues/489 I'm using commit 16cbda79167b4e2f2c6b61b218658f0f660d97f9 and it seems to work now.
(package! jupyter :pin "16cbda79167b4e2f2c6b61b218658f0f660d97f9")

[rustyCircleJones]

I believe the issue is how Doom Emacs does lazy loading of the org-babel libraries.

During normal initialization the org-jupyter uses org-mode-hook to execute org-babel-jupyter-aliases-from-kernelspecs. This function queries jupyter to know what kernels are installed. This is to deal with issues were people have a virtualenv-type projects setup were the kernels are not always available.

But because Doom Emacs doesn't have org-jupyter stuff loaded before org mode is activated the hooks are not available. The jupyter stuff doesn't get loaded until you try to execute a jupyter source block in a org file.

And since it doesn't have a list of kernels loaded it doesn't know what to do. Hence the:

org-babel-execute-src-block: No org-babel-execute function for jupyter-python!

error.

If you execute the source block, let it fail, then M-: (org-babel-jupyter-aliases-from-kernelspecs) it will work correctly after that.

[rustyCircleJones]

(with-eval-after-load 'ob-jupyter
 (org-babel-jupyter-aliases-from-kernelspecs))

Adding this to my doom/config.el file and running 'doom sync' gets org-babel with jupyter working for me. Just tested this on the latest 'doom upgrade' with no pinning of related packages.

[g6ai]

The org-babel-jupyter-aliases-from-kernelspecs solution works on the second execution and onwards. The (advice-remove #'org-babel-do-load-languages #'ignore) solution as mentioned emacs-jupyter/jupyter#306 (comment) and emacs-jupyter/jupyter#489 (comment) works on the first execution, but invokes a check of Python env if you are using Doom's direnv module when you open any org file. The former workaround seems to be less messy.