nbsphinx 0.9 claims json problem reading notebooks

[henryiii]

Notebooks started reporting read errors on the 0.9 update.

scikit-hep/boost-histogram#836 (See https://readthedocs.org/projects/boost-histogram/builds/19766997/)

Another one:

https://readthedocs.org/projects/vector/builds/19756260/

Traceback:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbsphinx/__init__.py", line 613, in parse
    rststring, resources = exporter.from_notebook_node(nb, resources)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbsphinx/__init__.py", line 400, in from_notebook_node
    nb, resources, **kw)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 421, in from_notebook_node
    output = self.template.render(nb=nb_copy, resources=resources)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/jinja2/environment.py", line 1291, in render
    self.environment.handle_exception()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 131, in top-level template code
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/share/jupyter/nbconvert/templates/rst/index.rst.j2", line 1, in top-level template code
    {%- extends 'display_priority.j2' -%}
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/share/jupyter/nbconvert/templates/base/display_priority.j2", line 1, in top-level template code
    {%- extends 'base/null.j2' -%}
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/share/jupyter/nbconvert/templates/base/null.j2", line 26, in top-level template code
    {%- block body -%}
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/share/jupyter/nbconvert/templates/base/null.j2", line 29, in block 'body'
    {%- block body_loop -%}
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/share/jupyter/nbconvert/templates/base/null.j2", line 31, in block 'body_loop'
    {%- block any_cell scoped -%}
  File "<template>", line 17, in block 'any_cell'
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/share/jupyter/nbconvert/templates/base/null.j2", line 87, in block 'any_cell'
    {%- block markdowncell scoped-%} {%- endblock markdowncell -%}
  File "<template>", line 168, in block 'markdowncell'
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/share/jupyter/nbconvert/templates/rst/index.rst.j2", line 98, in block 'markdowncell'
    {{ cell.source | convert_pandoc("markdown", "rst") }}
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbsphinx/__init__.py", line 865, in convert_pandoc
    return markdown2rst(text)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbsphinx/__init__.py", line 1018, in markdown2rst
    rststring = pandoc(text, input_format, 'rst', filter_func=filter_func)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbsphinx/__init__.py", line 1056, in pandoc
    json_data = encode(filter_func(decode(json_data)))
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbsphinx/__init__.py", line 1008, in filter_func
    json_data = json.loads(text, object_hook=object_hook)
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/json/__init__.py", line 361, in loads
    return cls(**kw).decode(s)
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/cmd/build.py", line 281, in build_main
    app.build(args.force_all, args.filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/application.py", line 347, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 312, in build_update
    len(to_build))
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 326, in build
    updated_docnames = set(self.read())
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 433, in read
    self._read_serial(docnames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 454, in _read_serial
    self.read_doc(docname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/builders/__init__.py", line 510, in read_doc
    publisher.publish()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/docutils/core.py", line 218, in publish
    self.settings)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/sphinx/io.py", line 104, in read
    self.parse()
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/docs/checkouts/readthedocs.org/user_builds/boost-histogram/envs/836/lib/python3.7/site-packages/nbsphinx/__init__.py", line 624, in parse
    str(e))
nbsphinx.NotebookError: JSONDecodeError in notebooks/BoostHistogramHandsOn.ipynb:
Expecting value: line 1 column 1 (char 0)

Notebook error:
JSONDecodeError in notebooks/BoostHistogramHandsOn.ipynb:
Expecting value: line 1 column 1 (char 0)

[tomkralidis]

Similar issue here: https://readthedocs.org/projects/wis2box/builds/19768871/

[mgeier]

Thanks for reporting this!

The mentioned projects use an old RTD image which has a very old Pandoc version (AFAICT version 1.19.2.4), which doesn't yet support the smart option which I've disabled in #716.

I have created #721 to fix this issue.

However, you should probably take this opportunity to update your RTD settings, see https://docs.readthedocs.io/en/stable/config-file/v2.html

You should adapt your .readthedocs.yml to look something like this:

version: 2
build:
  os: ubuntu-22.04
  tools:
    python: "3"

[mgeier]

I have just released nbsphinx 0.9.1 which should fix this error.

I would still advise everyone to update their RTD settings!

[henryiii]

Agree that's the best solution! Just going to take some time (or some all-repos-grep magic...) ;)