json-read: JSON readtable error with python 3.6.3

[lengyueyang]

We I use python 3.6.2 and 3.6.2, when execute, occur json-read: JSON readtable error, but when use python 3.5.2, all is ok

[dcherian]

I had this problem and had to downgrade.

Can we have a section in README that mentions the latest version of python this has been tested on?

[andrewsanchez]

Neither downgrading with conda install python=3.5 (resulted in 3.5.4) nor conda install python=3.5.2 worked for me.

[dcherian]

Oh I meant downgrade ob-ipython. I haven't tried with python 3.5

[dcherian]

Can someone who's using the latest ob-ipython please comment with versions of python / ipython / jupyter they're using?

[nicholas-miklaucic]

I think I figured this out, at least on my system (latest version of ob-ipython, ipython version 6.1.0, python version 3.6.2 with Anaconda). I got this error too, and I fixed it by using exec-path-from-shell to set the PATH correctly. Install that from MELPA and put this in your init file before loading ob-ipython:

(when (memq window-system '(mac ns x))
  (exec-path-from-shell-initialize)) 

I think the reason this happens is because it uses jupyter to get a JSON output of the kernels, but it runs that via shell-command, which doesn't use the exec-path variable the rest of Emacs uses. You should be good if M-! jupyter -h returns a normal help entry and not zsh:1:command not found: 'jupyter' or equivalent.

This is probably a documentation problem moreso than a code problem, as I'm not sure how you're supposed to be able to divine where the jupyter executable is in elisp.

[dcherian]

Hmm... I've always had exec-path-from-shell configured and running. M-! jupyter -h returns a normal help entry too.

Is there a way to see the string json-read is having trouble decoding? My elisp isn't good enough to figure it out.

[nicholas-miklaucic]

We must have different errors with the same error message. Can you try invoking toggle-debug-on-errorand then running it, or if that doesn't work trying toggle-debug-on-entry with json-read?

[dcherian]

@nicholas-miklaucic Thanks!

Using debug-on-entry showed this (newlines added for clarity):

  json-read-from-string("/home/deepak/.bashrc: line 115: bind: warning: line editing not enabled\n
{\n 
 \"kernelspecs\": {\n    \"python3\": {\n     
 \"resource_dir\": \"/home/deepak/anaconda3/share/jupyter/kernels/python3\",\n      \"spec\": {\n        \"argv\": [\n         
 \"/home/deepak/anaconda3/bin/python\",\n          \"-m\",\n          \"ipykernel_launcher\",\n          \"-f\",\n          \"{connection_file}\"\n        ],\n        \"env\": {},\n       
 \"display_name\": \"Python 3\",\n        \"language\": \"python\"\n      }\n    }\n  }\n}\n")

Once I comment the line that makes bash throw a warning, things work!

@gregsexton Maybe a regexp to select the bit that begins with {\n and ends with \n") before passing it to json-read would do the trick? Perhaps you could catch the error and display the string so it's clear what json-read is choking on?

[georgek]

This isn't working at all for me. Using Python 3.6.2 and getting the same JSON read table error. This is me trying to run the example:

Debugger entered--Lisp error: (json-readtable-error)

  signal(json-readtable-error nil)
  json-read()
  (cons (json-read) acc)
  (setq acc (cons (json-read) acc))
  (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line))
  (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc))
  (let ((json-array-type (quote list))) (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc)))
  ob-ipython--collect-json()
  (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json))
  (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))
  (progn (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json))))
  (unwind-protect (progn (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))
  (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((ret (apply (quote call-process-region) code nil (ob-ipython--get-python) nil t nil (list "--" ob-ipython-client-path "--conn-file" name "--execute")))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer))))
  (let ((temp-buffer (generate-new-buffer " *temp*"))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (let ((ret (apply ... code nil ... nil t nil ...))) (if (> ret 0) (ob-ipython--dump-error (buffer-string)) (goto-char (point-min)) (ob-ipython--collect-json)))) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))
  ob-ipython--execute-request("%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" "default")
  (ob-ipython--eval (ob-ipython--execute-request (org-babel-expand-body:generic (encode-coding-string body (quote utf-8)) params (org-babel-variable-assignments:python params)) (ob-ipython--normalize-session session)))
  (let ((ret (ob-ipython--eval (ob-ipython--execute-request (org-babel-expand-body:generic (encode-coding-string body (quote utf-8)) params (org-babel-variable-assignments:python params)) (ob-ipython--normalize-session session))))) (if ret (progn (ob-ipython--process-response ret file result-type))))
  (let* ((file (cdr (assoc :ipyfile params))) (session (cdr (assoc :session params))) (result-type (cdr (assoc :result-type params)))) (ob-ipython--create-kernel (ob-ipython--normalize-session session) (cdr (assoc :kernel params))) (let ((ret (ob-ipython--eval (ob-ipython--execute-request (org-babel-expand-body:generic (encode-coding-string body ...) params (org-babel-variable-assignments:python params)) (ob-ipython--normalize-session session))))) (if ret (progn (ob-ipython--process-response ret file result-type)))))
  ob-ipython--execute-sync("%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" ((:colname-names) (:rowname-names) (:result-params "replace" "drawer") (:result-type . value) (:results . "replace drawer") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
  (if (cdr (assoc :async params)) (ob-ipython--execute-async body params) (ob-ipython--execute-sync body params))
  org-babel-execute:ipython("%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" ((:colname-names) (:rowname-names) (:result-params "replace" "drawer") (:result-type . value) (:results . "replace drawer") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
  org-babel-execute-src-block(nil ("ipython" "%matplotlib inline\nimport matplotlib.pyplot as plt\nimport numpy as np" ((:colname-names) (:rowname-names) (:result-params "drawer" "replace") (:result-type . value) (:results . "drawer replace") (:exports . "code") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 714 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)
  call-interactively(org-ctrl-c-ctrl-c nil nil)
  command-execute(org-ctrl-c-ctrl-c)

There are also seem to be problems with it killing the jupyter process. I have to kill it manually otherwise I get errors from ZMQ saying the port is already in use.

[nicholas-miklaucic]

Try invoking toggle-debug-on-entry with json-read to see what it's choking on: your error seems different as it's for running code, AFAIK.

[dcherian]

You'll also want to make sure that jupyter isn't writing any warnings also. I had to remove those too.

[georgek]

I get a buffer called *ob-ipython-kernel-default* with some output from starting a jupyter console. It seems that json-read is being run on this buffer which doesn't contain anything that looks like JSON:

Jupyter console 5.2.0

Python 3.6.2 (default, Sep 28 2017, 13:58:54) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: 

[dcherian]

@georgek That's how it looks like when it works.

Set debug-on-entry to stop at ob-ipython--collect-json. When it does hit c to continue, do this twice and it should show the string it's trying to read. That'll tell you what it's choking on

EDIT: Not sure that will work actually but you can try. I found the second answer here helpful: https://stackoverflow.com/questions/3257480/how-to-debug-elisp#3258205

[georgek]

I've tried using edebug and stepping through. The difficulty seems to be that in ob-ipython--execute-request it runs ob-ipython--collect-json in a temporary buffer. I can't figure out how to look at what's being written in the temp buffer, though.

[georgek]

OK, I finally figured it out. It's not easy to debug something that uses with-temp-buffer, but eventually I figured out I could manually repalce that with a real buffer and see the output. Turns out my ipython prints an error about being run in a virtualenv which is not proper JSON. So now I need to figure out how to suppress that warning (which I think is safe).

I succumbed to installing jupyter/ipython within each virtualenv and now there is no error and it works.

[nanounanue]

Any news about this bug?

[ludicludo]

Hi everybody,
I have the same issue with python 2.7.12.
@georgek : how to tell emacs to use a particular virtualenv installation of jupyter/python ? I am a bit confused at this point.

[nanounanue]

@ludicludo you can use pyenv-mode for specify the virtualenv.

[nanounanue]

Following the instructions in PR #148 I got:

If I execute M-! jupyter --version, I got 4.40 (so, no problem here), and then when I activate the debug-mode, I got the following message when I try to run any block, for example:

#+BEGIN_SRC ipython
print("hola")
#+END_SRC

* json-read()
  ob-ipython--collect-json()
  ob-ipython--execute-request("print(\"hola\")" "default")
  ob-ipython--execute-sync("print(\"hola\")" ((:colname-names) (:rowname-names) (:result-params "replace" "drawer") (:result-type . value) (:results . "replace drawer") (:exports . "both") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
  org-babel-execute:ipython("print(\"hola\")" ((:colname-names) (:rowname-names) (:result-params "replace" "drawer") (:result-type . value) (:results . "replace drawer") (:exports . "both") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
  #f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer.  Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1871d41>)(nil ("ipython" "print(\"hola\")" ((:colname-names) (:rowname-names) (:result-params "drawer" "replace") (:result-type . value) (:results . "drawer replace") (:exports . "both") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 9617 "(ref:%s)") nil)
  ob-async-org-babel-execute-src-block(#f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer.  Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1871d41>) nil ("ipython" "print(\"hola\")" ((:colname-names) (:rowname-names) (:result-params "drawer" "replace") (:result-type . value) (:results . "drawer replace") (:exports . "both") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 9617 "(ref:%s)"))
  apply(ob-async-org-babel-execute-src-block #f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer.  Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1871d41>) (nil ("ipython" "print(\"hola\")" ((:colname-names) (:rowname-names) (:result-params "drawer" "replace") (:result-type . value) (:results . "drawer replace") (:exports . "both") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 9617 "(ref:%s)")))
  org-babel-execute-src-block(nil ("ipython" "print(\"hola\")" ((:colname-names) (:rowname-names) (:result-params "drawer" "replace") (:result-type . value) (:results . "drawer replace") (:exports . "both") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 9617 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)
  call-interactively(org-ctrl-c-ctrl-c nil nil)
  #f(compiled-function (cmd &optional record-flag keys special) "Execute CMD as an editor command.\nCMD must be a symbol that satisfies the `commandp' predicate.\nOptional second arg RECORD-FLAG non-nil\nmeans unconditionally put this command in the variable `command-history'.\nOtherwise, that is done only if an arg is read using the minibuffer.\nThe argument KEYS specifies the value to use instead of (this-command-keys)\nwhen reading the arguments; if it is nil, (this-command-keys) is used.\nThe argument SPECIAL, if non-nil, means that this command is executing\na special event, so ignore the prefix argument and don't clear it." #<bytecode 0x249b73>)(org-ctrl-c-ctrl-c nil nil nil)
  ad-Advice-command-execute(#f(compiled-function (cmd &optional record-flag keys special) "Execute CMD as an editor command.\nCMD must be a symbol that satisfies the `commandp' predicate.\nOptional second arg RECORD-FLAG non-nil\nmeans unconditionally put this command in the variable `command-history'.\nOtherwise, that is done only if an arg is read using the minibuffer.\nThe argument KEYS specifies the value to use instead of (this-command-keys)\nwhen reading the arguments; if it is nil, (this-command-keys) is used.\nThe argument SPECIAL, if non-nil, means that this command is executing\na special event, so ignore the prefix argument and don't clear it." #<bytecode 0x249b73>) org-ctrl-c-ctrl-c)
  apply(ad-Advice-command-execute #f(compiled-function (cmd &optional record-flag keys special) "Execute CMD as an editor command.\nCMD must be a symbol that satisfies the `commandp' predicate.\nOptional second arg RECORD-FLAG non-nil\nmeans unconditionally put this command in the variable `command-history'.\nOtherwise, that is done only if an arg is read using the minibuffer.\nThe argument KEYS specifies the value to use instead of (this-command-keys)\nwhen reading the arguments; if it is nil, (this-command-keys) is used.\nThe argument SPECIAL, if non-nil, means that this command is executing\na special event, so ignore the prefix argument and don't clear it." #<bytecode 0x249b73>) org-ctrl-c-ctrl-c)
  command-execute(org-ctrl-c-ctrl-c)

So, that doesn't seem to fix the problem

[nicholas-miklaucic]

Can you try doing the toggle-debug-on-error thing I mentioned above this? It looks like it's choking on something that's supposed to be JSON, but you can't actually see what the JSON is in your example.

[nanounanue]

@nicholas-miklaucic After M-x toggle-debug-on-error, tried to execute the same block and I got

Debugger entered--Lisp error: (json-readtable-error . 27)
  signal(json-readtable-error 27)
  json-read()
  ob-ipython--collect-json()
  ob-ipython--execute-request("print(\"Hola\")" "default")
  ob-ipython--execute-sync("print(\"Hola\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
  org-babel-execute:ipython("print(\"Hola\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:session)))
  #f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer.  Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1071ab5>)(nil ("ipython" "print(\"Hola\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 8235 "(ref:%s)") nil)
  ob-async-org-babel-execute-src-block(#f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer.  Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1071ab5>) nil ("ipython" "print(\"Hola\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 8235 "(ref:%s)"))
  apply(ob-async-org-babel-execute-src-block #f(compiled-function (&optional arg info params) "Execute the current source code block.\nInsert the results of execution into the buffer.  Source code\nexecution and the collection and formatting of results can be\ncontrolled through a variety of header arguments.\n\nWith prefix argument ARG, force re-execution even if an existing\nresult cached in the buffer would otherwise have been returned.\n\nOptionally supply a value for INFO in the form returned by\n`org-babel-get-src-block-info'.\n\nOptionally supply a value for PARAMS which will be merged with\nthe header arguments specified at the front of the source code\nblock." (interactive nil) #<bytecode 0x1071ab5>) (nil ("ipython" "print(\"Hola\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 8235 "(ref:%s)")))
  org-babel-execute-src-block(nil ("ipython" "print(\"Hola\")" ((:colname-names) (:rowname-names) (:result-params "replace") (:result-type . value) (:results . "replace") (:exports . "code") (:session) (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 8235 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)
  call-interactively(org-ctrl-c-ctrl-c nil nil)
  #f(compiled-function (cmd &optional record-flag keys special) "Execute CMD as an editor command.\nCMD must be a symbol that satisfies the `commandp' predicate.\nOptional second arg RECORD-FLAG non-nil\nmeans unconditionally put this command in the variable `command-history'.\nOtherwise, that is done only if an arg is read using the minibuffer.\nThe argument KEYS specifies the value to use instead of (this-command-keys)\nwhen reading the arguments; if it is nil, (this-command-keys) is used.\nThe argument SPECIAL, if non-nil, means that this command is executing\na special event, so ignore the prefix argument and don't clear it." #<bytecode 0x249b73>)(org-ctrl-c-ctrl-c nil nil nil)
  ad-Advice-command-execute(#f(compiled-function (cmd &optional record-flag keys special) "Execute CMD as an editor command.\nCMD must be a symbol that satisfies the `commandp' predicate.\nOptional second arg RECORD-FLAG non-nil\nmeans unconditionally put this command in the variable `command-history'.\nOtherwise, that is done only if an arg is read using the minibuffer.\nThe argument KEYS specifies the value to use instead of (this-command-keys)\nwhen reading the arguments; if it is nil, (this-command-keys) is used.\nThe argument SPECIAL, if non-nil, means that this command is executing\na special event, so ignore the prefix argument and don't clear it." #<bytecode 0x249b73>) org-ctrl-c-ctrl-c)
  apply(ad-Advice-command-execute #f(compiled-function (cmd &optional record-flag keys special) "Execute CMD as an editor command.\nCMD must be a symbol that satisfies the `commandp' predicate.\nOptional second arg RECORD-FLAG non-nil\nmeans unconditionally put this command in the variable `command-history'.\nOtherwise, that is done only if an arg is read using the minibuffer.\nThe argument KEYS specifies the value to use instead of (this-command-keys)\nwhen reading the arguments; if it is nil, (this-command-keys) is used.\nThe argument SPECIAL, if non-nil, means that this command is executing\na special event, so ignore the prefix argument and don't clear it." #<bytecode 0x249b73>) org-ctrl-c-ctrl-c)
  command-execute(org-ctrl-c-ctrl-c)

[nanounanue]

Also, I am getting two buffers: ob-ipython-out (empty) and
*ob-ipython-kernel-default* with the following:

Jupyter console 5.2.0

Python 3.6.2 (default, Dec  6 2017, 13:55:54) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.



In [1]: 

[nicholas-miklaucic]

I have jupyter 4.3.0. Perhaps that's important? If it is, this belongs as a separate bug.

…

On Tue, Dec 26, 2017 at 6:53 PM Adolfo De Unánue ***@***.***> wrote: Also, I am getting two buffers: ob-ipython-out (empty) and *ob-ipython-kernel-default* with the following: Jupyter console 5.2.0 Python 3.6.2 (default, Dec 6 2017, 13:55:54) Type 'copyright', 'credits' or 'license' for more information IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help. In [1]: — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#135 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZLD8c82XnZ93FbotCIfng5NqX54lJcYks5tEYcCgaJpZM4QMIhv> .

[nanounanue]

Actually my jupyter version is 4.4.0, the jupyter-console is 5.2.0 Which is your jupyter-console version? @nicholas-miklaucic

[nicholas-miklaucic]

5.2., my bad: didn't realize the version counters were different.

…

On Wed, Dec 27, 2017 at 1:04 AM Adolfo De Unánue ***@***.***> wrote: Actually my jupyter version is 4.4.0, the jupyter-console is 5.2.0 Which is your jupyter-console version? @nicholas-miklaucic <https://github.com/nicholas-miklaucic> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#135 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZLD8RtXC9uuF6VHfI8QfbIaFHUAhgFYks5tEd3rgaJpZM4QMIhv> .

[nanounanue]

I am so confused with this error... could you share with me your configuration files for org-mode ? Maybe the problem is there, or maybe is the ob-ipython version?

[dcherian]

@nanounanue You could replace with-temp-buffer with with-temp-file, that should show you what it's choking on. Ref: #135 (comment)

[nanounanue]

@dcherian What do you mean? Could you be more specific?

[aayusuff]

You can set ob-ipython-command to the path of jupyter. e,g
(setq ob-ipython-command "/home/name/miniconda3/bin/jupyter") . You can then restart emacs.

[nanounanue]

@aayusuff I am using pyenv-mode for that. ob-ipython is picking the right path. But I will try your suggestion anyway.

[ghost]

Hi everyone, I'm trying to use ob-ipython in a virtualenv environment. And I get this error too.
I think I found the problem.

#+begin_src ipython :results output drawer
print("hello")
#+end_src

After you execute code in ob-ipython, you get the error

Debugger entered--Lisp error: (json-readtable-error 27)
signal(json-readtable-error (27))
json-read()
ob-ipython--collect-json()
ob-ipython--execute-request("print(\"hello\")" "none")
....

And I print the buffer when using ob-ipython--execute-request, I got this:

executing Ipython code block...
�]0;IPython: fun/test�{"header": {"version": "5.3", "date": "2018-06-14 22:38:43.332538+00:00", "session": "77e98729-7084772c31ba8a39c3214ef2", "username": "zwild", "msg_type": "status", "msg_id": "9982f44e-f208256d22f6bce8e6d9393e"}, "msg_id": "9982f44e-f208256d22f6bce8e6d9393e", "msg_type": "status", "parent_header": {"version": "5.3", "date": "2018-06-14 22:38:43.329612+00:00", "session": "f6fbcc63-b051c0a706525968907bc55a", "username": "zwild", "msg_type": "execute_request", "msg_id": "a369e270-923588add6c9099c7e6396d4"}, "metadata": {}, "content": {"execution_state": "busy"}, "buffers": [], "channel": "io"}
{"header": {"version": "5.3", "date": "2018-06-14 22:38:43.332983+00:00", "session": "77e98729-7084772c31ba8a39c3214ef2", "username": "zwild", "msg_type": "execute_input", "msg_id": "cb651797-0adb557949dc7a2b05e4e549"}, "msg_id": "cb651797-0adb557949dc7a2b05e4e549", "msg_type": "execute_input", "parent_header": {"version": "5.3", "date": "2018-06-14 22:38:43.329612+00:00", "session": "f6fbcc63-b051c0a706525968907bc55a", "username": "zwild", "msg_type": "execute_request", "msg_id": "a369e270-923588add6c9099c7e6396d4"}, "metadata": {}, "content": {"code": "print(\"hello\")", "execution_count": 4}, "buffers": [], "channel": "io"}
{"header": {"version": "5.3", "date": "2018-06-14 22:38:43.334873+00:00", "session": "77e98729-7084772c31ba8a39c3214ef2", "username": "zwild", "msg_type": "stream", "msg_id": "efc54877-814066e39372434d4f54cefc"}, "msg_id": "efc54877-814066e39372434d4f54cefc", "msg_type": "stream", "parent_header": {"version": "5.3", "date": "2018-06-14 22:38:43.329612+00:00", "session": "f6fbcc63-b051c0a706525968907bc55a", "username": "zwild", "msg_type": "execute_request", "msg_id": "a369e270-923588add6c9099c7e6396d4"}, "metadata": {}, "content": {"name": "stdout", "text": "hello\n"}, "buffers": [], "channel": "io"}
{"header": {"version": "5.3", "date": "2018-06-14 22:38:43.337765+00:00", "session": "77e98729-7084772c31ba8a39c3214ef2", "username": "zwild", "msg_type": "status", "msg_id": "cb44727a-bb15beededb4f210f3de056b"}, "msg_id": "cb44727a-bb15beededb4f210f3de056b", "msg_type": "status", "parent_header": {"version": "5.3", "date": "2018-06-14 22:38:43.329612+00:00", "session": "f6fbcc63-b051c0a706525968907bc55a", "username": "zwild", "msg_type": "execute_request", "msg_id": "a369e270-923588add6c9099c7e6396d4"}, "metadata": {}, "content": {"execution_state": "idle"}, "buffers": [], "channel": "io"}
{"header": {"version": "5.3", "date": "2018-06-14 22:38:43.336724+00:00", "session": "77e98729-7084772c31ba8a39c3214ef2", "username": "zwild", "msg_type": "execute_reply", "msg_id": "81cff026-cfa4fa88da630c3b71ad2aa5"}, "msg_id": "81cff026-cfa4fa88da630c3b71ad2aa5", "msg_type": "execute_reply", "parent_header": {"version": "5.3", "date": "2018-06-14 22:38:43.329612+00:00", "session": "f6fbcc63-b051c0a706525968907bc55a", "username": "zwild", "msg_type": "execute_request", "msg_id": "a369e270-923588add6c9099c7e6396d4"}, "metadata": {"started": "2018-06-14T22:38:43.332929Z", "dependencies_met": true, "engine": "f65ba4ef-95d1-4c18-aba2-12728b8ec990", "status": "ok"}, "content": {"status": "ok", "execution_count": 4, "user_expressions": {}, "payload": []}, "buffers": [], "channel": "shell"}

And there are some strange characters �]0;IPython: fun/test�.

I think the problem is here. Json can't load the temp buffer because of there strange characters.

So I hack these code.

(defun ob-ipython--collect-json ()
  ;; hacks here
  (when (re-search-forward "{" nil t)
    (backward-char))
  ;; hacks end
  (let ((json-array-type 'list))
    (let (acc)
      (while (not (= (point) (point-max)))
        (setq acc (cons (json-read) acc))
        (forward-line))
      (nreverse acc))))

or

(advice-add 'ob-ipython--collect-json :before
            (lambda (&rest args)
              (when (re-search-forward "{" nil t)
                (backward-char))))

I got the result

#+RESULTS:
:RESULTS:
hello
:END:

There is a little hack. Now you can use any virtualenv package which set PATH, exec-path and eshell-path-env correct. For example pyenv and virtualenvwrapper.el.

The problem is the strange characters. Any hack like this or some code that clean the characters will solve the problem.
I think someone can make this code better and please make a pull request.

[theottm]

Hello everyone,

when I use ob-ipython in Emacs launched from a shell with an active virtualenv (using pyenv) I get the same json-readtable error. My code runs on the kernel though, I could verify it by using the interactive session with C-c C-v C-z. However I don't get any output in the result block.

Now when I launch Emacs from my basic system environment and then use pyen-mode-set to activate the exact same virtualenv from within Emacs, everything works fine.

I am happy with that but still curious to understand what exactly is going wrong.
I added @dangirsh 's fix dangirsh@df13b94 to the ob-iython--execute-request function to be able to have a look at the json ob-ipython and the kernel use to communicate and I finally found where the problem was in my case : the kernel throws a warning message that is not json formated.

/home/teddd/.local/lib/python3.5/site-packages/IPython/core/interactiveshell.py:763: UserWarning: Attempting to work in a virtualenv. If you encounter problems, please install IPython inside the virtualenv. warn("Attempting to work in a virtualenv. If you encounter problems, please "

Thus when reading this message, ob-ipython--collect-json returns an error number being the first letter encountered in the error message in decimal form. (In my case error 47 corresponds to the "/" character) This explains why this number is so random and cryptic.

As I see it the problem is not only virtualenvs, but also the way ob-ipython handles outputs coming from the kernel. This non json message should clearly be redirected to the error/warning buffer. Maybe there should be a filter ? Is there already such a function somewhere else in the code ? How are error messages normaly handled @gregsexton ?

Of course the simplest way would be to just run pyenv or get the PATH from the shell and not have any error message, but this wouldn't make the program more robust. Who knows if the next version of Jupyter doesn't throw wierd stuff on our poor json-read ?

[jchkoch]

Hello everyone,

Currently, my ob-ipython setup returned to having similar error as discussed in this issue when I attempt to run a ipython code block from within a org mode file. If others think I should open a new issue, I will do so; however, after reading the comments under this issue I felt that my error was similar enough to warrant posting a comment here.

Now to the actual error message I am seeing:

error in process sentinel: json-read: JSON readtable errorxo error in process sentinel: JSON readtable error

When toggling debug on error, I get the following backtrace:

Debugger entered--Lisp error: (json-readtable-error)
  signal(json-readtable-error nil)
  json-read()
  (cons (json-read) acc)
  (setq acc (cons (json-read) acc))
  (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line))
  (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc))
  (let ((json-array-type (quote list))) (let (acc) (while (not (= (point) (point-max))) (setq acc (cons (json-read) acc)) (forward-line)) (nreverse acc)))
  ob-ipython--collect-json()
  (ob-ipython--eval (ob-ipython--collect-json))
  (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9))
  (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))
  (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9))))
  (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async))
  (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async)))
  (lambda (G9 G10 proc state) (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async))))(--args-- --callback-- #<process execute> "finished\n")
  apply((lambda (G9 G10 proc state) (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer (process-buffer proc)) (goto-char (point-min)) (apply (symbol-value G10) (cons (ob-ipython--eval (ob-ipython--collect-json)) (symbol-value G9)))) (ob-ipython--maybe-run-async)))) --args-- --callback-- (#<process execute> "finished\n"))
  (lambda (&rest --cl-rest--) (apply (quote (lambda (G9 G10 proc state) (if (not (process-live-p proc)) (progn (save-current-buffer (set-buffer ...) (goto-char ...) (apply ... ...)) (ob-ipython--maybe-run-async))))) (quote --args--) (quote --callback--) --cl-rest--))(#<process execute> "finished\n")

Furthermore, the following is additional information regarding package versions in my current setup:

jupyter-console --version 5.2.0
jupyter --version 4.4.0
ipython --version 6.4.0
python --version Python 3.6.4

emacs --version
GNU Emacs 25.3.1
Copyright (C) 2017 Free Software Foundation, Inc.
GNU Emacs comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GNU Emacs
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING.

I have the most recent version of ob-ipython installed directly from the git repository and I am running a openSUSE Leap 15.0 linux machine.
I have tried some of the solutions proposed in this issue to workaround this error but none of them worked in my case. The only thing that helped is to add (exec-path-from-shell-initialize) in my emacs configuration before loading ob-ipython which only caused no backtrace to be generated while still displaying the error json-read: JSON readtable error.

If anyone has any ideas for how I can fix this issue I am experiencing, I would appreciate the help.

[theottm]

@jchkoch your error seem to be different, but the problem might be the same. The problem is that this error message doesn't tell a lot about where the problem reside. Generally though it is a problem of the way ob-ipython and your python environment interact.

Have you checked if the env is well set in your org file ? You can try shell-command RET python --version RET and the others. If there is a problem here, using pyenv and its emacs plugin might be a solution.

To address your problem more specifically, I advice you to use this hack dangirsh@df13b94 to be able to have a look at the json and understand what in it causes the reader to crash.

I also had good results using edebug. It is implemented in Emacs and allows you to follow step by step the program in the elisp source code, knowing what data is coming in and out of each step. You could thus see more precisely where it crashes in the json-read function. But my guess is just that the json is badly formatted, have a look a it first :)

Good luck !

[jchkoch]

The json-readtable error that I reported 2 days ago (#135 (comment)) disappeared when I restarted emacs today. So far, I have not ascertained why the issue disappeared. Anyway, thanks for the reply @theottm and I will keep this in mind if the error ever comes back.

[theottm]

glad to hear it !

[ganesh-krishnan]

I was seeing the same error. #135 (comment) worked. Thank you!

[thcoffee-msft]

I was getting a similar problem, but using edebug revealed that I was getting extra messages after the JSON response rather than before. I extended the solution in #135 (comment) to remove anything around or between top-level JSON objects in the response, which seems to be working for me:

; kill extraneous content in ipython returns between top-level JSON objects
(advice-add 'ob-ipython--collect-json :before
            (lambda (&rest args)
              (let ((start (point)))
                (set-mark (point))
                (while (re-search-forward "{" nil t)
                  (backward-char)
                  (kill-region (region-beginning) (region-end))
                  (re-search-forward "}\n" nil t)
                  (set-mark (point)))
                (end-of-buffer)
                (kill-region (region-beginning) (region-end))
                (goto-char start))))