python org-babel with :session header

[wangkev]

What did you expect to happen?
Have org-babel with python and :session header work and output result. Seems like the default org-babel python script that's run with the :session header doesn't always get interpreted properly.

What actually happened?
Had an indentation error (see below), and nothing was output.

Additional details:
Confirmed org-babel works as expected in vanilla Emacs and with doom, but without the org module. Also this seems to only be an issue when I include :session header. Sometimes if I execute the org-babel block again, it will work (i.e. org-babel outputs, and no indentation/parsing issue in *Python* buffer).

Best resources I could find so far:

• https://code.orgmode.org/jackkamm/org-mode/commit/c2fff65f36141b78d91af9d6264d0f936ee5a3a1
• https://emacs.stackexchange.com/questions/41336/src-python-in-org-mode-indentationerror-unexpected-indent

config.el (although have tried with the below excluded too):

(setq python-shell-interpreter "python3")

*Python* buffer:

import ast
Python 3.7.6 (default, Dec 30 2019, 19:38:28) 
[Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
try:
>>> >>> ...   File "<stdin>", line 2
    import codecs, os;__pyfile = codecs.open('''/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/pybvFjOf''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/pybvFjOf''');exec(compile(__code, '''/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/pybvFjOf''', 'exec'));
         ^
IndentationError: expected an indented block
>>>     with open('/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/babel-8x5dXm/python-RVrgqc') as f:
  File "<stdin>", line 1
    with open('/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/babel-8x5dXm/python-RVrgqc') as f:
    ^
IndentationError: unexpected indent
        __org_babel_python_ast = ast.parse(f.read())
>>>     __org_babel_python_final = __org_babel_python_ast.body[-1]
  File "<stdin>", line 1
    __org_babel_python_ast = ast.parse(f.read())
    ^
IndentationError: unexpected indent
    if isinstance(__org_babel_python_final, ast.Expr):
>>>         __org_babel_python_ast.body = __org_babel_python_ast.body[:-1]
  File "<stdin>", line 1
    __org_babel_python_final = __org_babel_python_ast.body[-1]
    ^
IndentationError: unexpected indent
        exec(compile(__org_babel_python_ast, '<string>', 'exec'))
>>>         __org_babel_python_final = eval(compile(ast.Expression(
  File "<stdin>", line 1
    if isinstance(__org_babel_python_final, ast.Expr):
    ^
IndentationError: unexpected indent
            __org_babel_python_final.value), '<string>', 'eval'))
>>>     else:
  File "<stdin>", line 1
    __org_babel_python_ast.body = __org_babel_python_ast.body[:-1]
    ^
IndentationError: unexpected indent
        exec(compile(__org_babel_python_ast, '<string>', 'exec'))
>>>         __org_babel_python_final = None
  File "<stdin>", line 1
    exec(compile(__org_babel_python_ast, '<string>', 'exec'))
    ^
IndentationError: unexpected indent
except Exception:
>>>     from traceback import format_exc
  File "<stdin>", line 1
    __org_babel_python_final = eval(compile(ast.Expression(
    ^
IndentationError: unexpected indent
    __org_babel_python_final = format_exc()
>>>     raise
  File "<stdin>", line 1
    __org_babel_python_final.value), '<string>', 'eval'))
    ^
IndentationError: unexpected indent

>>> open('/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/babel-8x5dXm/python-JoOoC9', 'w').write(str(__org_babel_python_final))
  File "<stdin>", line 1
    else:
    ^
IndentationError: unexpected indent

>>> 
  File "<stdin>", line 1
    exec(compile(__org_babel_python_ast, '<string>', 'exec'))
    ^
IndentationError: unexpected indent
'org_babel_python_eoe'
>>>   File "<stdin>", line 1
    __org_babel_python_final = None
    ^
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    except Exception:
         ^
SyntaxError: invalid syntax
>>>   File "<stdin>", line 1
    from traceback import format_exc
    ^
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    __org_babel_python_final = format_exc()
    ^
IndentationError: unexpected indent
>>>   File "<stdin>", line 1
    raise
    ^
IndentationError: unexpected indent
>>> >>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name '__org_babel_python_final' is not defined
>>> >>> >>> 'org_babel_python_eoe'
>>> 

Then after a few tries, the python code will be interpreted fine:

>>> import ast
>>> try:
...     with open('/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/babel-s9dZu3/python-XufMTg') as f:
...         __org_babel_python_ast = ast.parse(f.read())
...     __org_babel_python_final = __org_babel_python_ast.body[-1]
...     if isinstance(__org_babel_python_final, ast.Expr):
...         __org_babel_python_ast.body = __org_babel_python_ast.body[:-1]
...         exec(compile(__org_babel_python_ast, '<string>', 'exec'))
...         __org_babel_python_final = eval(compile(ast.Expression(
...             __org_babel_python_final.value), '<string>', 'eval'))
...     else:
...         exec(compile(__org_babel_python_ast, '<string>', 'exec'))
...         __org_babel_python_final = None
... except Exception:
...     from traceback import format_exc
...     __org_babel_python_final = format_exc()
...     raise
... 
open('/var/folders/13/x5h4fyvj3c715d6pnbd68ypm0000gn/T/babel-s9dZu3/python-Hy9Zle', 'w').write(str(__org_babel_python_final))

Steps to reproduce:

1. Install vanilla emacs
2. Confirm org-babel works in vanilla emacs. Minimal repro:
   2a. In scratch buffer, change to org-mode (M-x org-mode)
   2b. Paste below two source blocks:

#+begin_src emacs-lisp
(org-babel-do-load-languages
 'org-babel-load-languages
 '((python . t)))
#+end_src

#+begin_src python :session :results raw
5
#+end_src

2c. Execute blocks and confirm output:

#+results:
5

2. Install doom
3. Enable org in doom config
4. Follow 2a to 2c to see no results and aforementioned traceback in *Python* buffer.

System information:

```
emacs   version    27.0.91
        features   NOTIFY KQUEUE ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS PDUMPER GMP
        build      May 20, 2020
        buildopts  (--enable-locallisppath=/usr/local/share/emacs/site-lisp --infodir=/usr/local/Cellar/emacs-mac/HEAD-b71b8b1/share/info/emacs --prefix=/usr/local/Cellar/emacs-mac/HEAD-b71b8b1 --with-mac --enable-mac-app=/usr/local/Cellar/emacs-mac/HEAD-b71b8b1 --with-gnutls --with-modules)
        windowsys  mac
        daemonp    server-running
doom    version    2.0.9
        build      grafted, HEAD -> develop, origin/develop, origin/HEAD 94fc6f7 2020-05-20 01:10:15 -0400
        dir        ~/.dotfiles/emacs/.config/doom/
system  type       darwin
        config     x86_64-apple-darwin18.7.0
        shell      /bin/zsh
        uname      Darwin 18.7.0 Darwin Kernel Version 18.7.0: Mon Feb 10 21:08:45 PST 2020; root:xnu-4903.278.28~1/RELEASE_X86_64 x86_64
        path       (~/.virtualenvs/py3/bin /usr/local/bin /usr/bin /bin /usr/sbin /sbin /Library/TeX/texbin ~/.config/emacs/bin)
config  envfile    envvar-file
        elc-files  0
        modules    (:completion company ivy :ui deft doom doom-dashboard fill-column hl-todo minimap modeline ophints (popup +defaults) vc-gutter vi-tilde-fringe workspaces :editor (evil +everywhere) file-templates fold snippets :emacs dired electric ibuffer undo vc :checkers syntax spell :tools (eval +overlay) lookup magit :lang emacs-lisp markdown (org +roam) (python +lsp) sh :config literate (default +bindings +smartparens))
        packages   (n/a)
        unpin      (n/a)
        elpa       (n/a)
```

[flowersw]

I think add this to my config this may have solved it for me (setq org-babel-python-command "python3") seen in https://tecosaur.github.io/emacs-config/config.html#babel,code--1

[gpetrini]

I have the same error. I've tried @flowersw proposal and still got the same result. Is this something related to org_babel_python_eoe? Here is my doom configuration link

Additionally, it works in other org-files and in vanila emacs. Any clue if it is a python or org issue?

Update: I fixed adding :results outputto the header as follows

#+PROPERTY: header-args:python  :session *Turma_A* :exports results :results output :python /usr/bin/python3  

[hlissner]

@wangkev Is this still an issue? I cannot reproduce it. Please make sure you are on the latest version of Doom and on Emacs 27.1+, not 27.0.50 or earlier.

To change the interpreter python blocks in org-mode use change org-babel-python-command instead. Doom sets it to python-shell-interpreter, but it's possible you're setting it in such a way that it gets set too early; check its value with M-x helpful-variable RET org-babel-python-command).

[wangkev]

I haven't tried to use the session tag since raising the issue, but re-tested just now and do not have the same error anymore (granted on a different machine). Using emacs 27.1 and doom 2.0.9.

Happy to close this issue, or can leave it open for others who appear to have the same issue?

[hlissner]

It must've been fixed at some point then. Thanks for letting me know!