NotJSONError exception when translate notebook into Chinese with sphinx-intl

[seisman]

I'm trying to translate a English documentation into Chinese.

I use the sphinx utility sphinx-intl to do the translation. Everything works fine when I translate strings from rst files. However, the NotJSONError exception occurs if I translate any strings from IPython notebook:

Exception occurred:
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/nbformat/reader.py", line 17, in parse_json
    raise NotJSONError(("Notebook does not appear to be JSON: %r" % s)[:77] + "...")
nbformat.reader.NotJSONError: Notebook does not appear to be JSON: 'notebook中的语句'...
The full traceback has been saved in /tmp/sphinx-err-wv4oyu0l.log, if you want to report the issue to the developers.

The full information in the log /tmp/sphinx-err-wv4oyu0l.log is:

# Sphinx version: 1.6.5
# Python version: 3.6.3 (CPython)
# Docutils version: 0.14 
# Jinja2 version: 2.10
# Last messages:
#   building [html]: targets for 2 source files that are out of date
#   
#   updating environment:
#   
#   2 added, 0 changed, 0 removed
#   
#   reading sources... [ 50%] index
#   
#   reading sources... [100%] notebook
#   
# Loaded extensions:
#   alabaster (0.7.10) from /home/seisman/.anaconda/lib/python3.6/site-packages/alabaster/__init__.py
#   nbsphinx (0.2.18) from /home/seisman/.anaconda/lib/python3.6/site-packages/nbsphinx.py
Traceback (most recent call last):
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/nbformat/reader.py", line 14, in parse_json
    nb_dict = json.loads(s, **kwargs)
  File "/home/seisman/.anaconda/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/home/seisman/.anaconda/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/seisman/.anaconda/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/cmdline.py", line 306, in main
    app.build(opts.force_all, filenames)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/application.py", line 339, in build
    self.builder.build_update()
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 331, in build_update
    'out of date' % len(to_build))
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 344, in build
    updated_docnames = set(self.env.update(self.config, self.srcdir, self.doctreedir))
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 584, in update
    self._read_serial(docnames, self.app)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 603, in _read_serial
    self.read_doc(docname, app)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/environment/__init__.py", line 726, in read_doc
    pub.publish()
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/docutils/core.py", line 218, in publish
    self.apply_transforms()
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/docutils/core.py", line 199, in apply_transforms
    self.document.transformer.apply_transforms()
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/transforms/i18n.py", line 128, in apply
    node.line, self.config, settings)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/transforms/i18n.py", line 62, in publish_msgstr
    settings=settings,
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/sphinx/io.py", line 74, in read
    self.parse()
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/nbsphinx.py", line 623, in parse
    nb = nbformat.reads(inputstring, as_version=_ipynbversion)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/nbformat/__init__.py", line 74, in reads
    nb = reader.reads(s, **kwargs)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/nbformat/reader.py", line 58, in reads
    nb_dict = parse_json(s, **kwargs)
  File "/home/seisman/.anaconda/lib/python3.6/site-packages/nbformat/reader.py", line 17, in parse_json
    raise NotJSONError(("Notebook does not appear to be JSON: %r" % s)[:77] + "...")
nbformat.reader.NotJSONError: Notebook does not appear to be JSON: 'notebook中的语句'...

Below are five steps to reproduce this issue. I also created a minimal example with one commit for each step.

1. seisman/nbsphinx-bug-report@b2d030e: use sphinx-quickstart to create a new project
2. seisman/nbsphinx-bug-report@59b152e
   : add a notebook in English into the documentation. The build succeeds.
3. seisman/nbsphinx-bug-report@68f2b67: add basic sphinx-intl support and generate .po files. Both make html and make html-zh_CN succeed.
4. seisman/nbsphinx-bug-report@71fa220: translate strings from rst files into Chinese. Both make html and make html-zh_CN succeed.
5. seisman/nbsphinx-bug-report@13f16c4: translate strings from IPython notebook into Chinese. make html succeeds, and make html-zh_CN fails.

Any ideas how to fix this?

[mgeier]

Thanks for the detailed report!

Sadly, I don't know anything about Sphinx translations, but I've asked at the Sphinx mailing list: https://groups.google.com/forum/#!topic/sphinx-dev/PvAQfZcDeHw.

[seisman]

Thanks for your quick response. Hopefully someone can give an answer.

[mgeier]

I've created #156 which should hopefully fix this issue.

Note, however, that the use of reST strings is temporary and at some point in the future you might have to change your translations to use Markdown/CommonMark formatting.

[seisman]

It works as expected. Thanks for your working.

[mgeier]

@seisman Thanks for the feedback!

I've merged #156 and I will try to make a new release in the next few days.