Translated docs builds fail often with KeyError

[timgraham]

# Sphinx version: 1.3.4
# Python version: 3.4.0 (CPython)
# Docutils version: 0.12 release
# Jinja2 version: 2.7.3
# Last messages:
#   reading sources... [ 17%] internals/security
#   reading sources... [ 17%] internals/team
#   reading sources... [ 17%] intro/contributing
#   reading sources... [ 18%] intro/index
#   reading sources... [ 18%] intro/install
#   reading sources... [ 18%] intro/overview
#   reading sources... [ 18%] intro/reusable-apps
#   reading sources... [ 19%] intro/tutorial01
#   reading sources... [ 19%] intro/tutorial02
#   reading sources... [ 19%] intro/tutorial03
# Loaded extensions:
#   sphinx.ext.intersphinx (1.3.4) from /home/www/www/venv/lib/python3.4/site-packages/sphinx/ext/intersphinx.py
#   ticket_role (unknown version) from /home/www/www/data/docbuilds/1.8/docs/_ext/ticket_role.py
#   djangodocs (unknown version) from /home/www/www/data/docbuilds/1.8/docs/_ext/djangodocs.py
#   alabaster (0.7.7) from /home/www/www/venv/lib/python3.4/site-packages/alabaster/__init__.py
#   sphinx.ext.viewcode (1.3.4) from /home/www/www/venv/lib/python3.4/site-packages/sphinx/ext/viewcode.py
Traceback (most recent call last):
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/cmdline.py", line 244, in main
    app.build(opts.force_all, filenames)
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/application.py", line 266, in build
    self.builder.build_update()
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/builders/__init__.py", line 251, in build_update
    'out of date' % len(to_build))
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/builders/__init__.py", line 265, in build
    self.doctreedir, self.app))
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/environment.py", line 618, in update
    self._read_serial(docnames, app)
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/environment.py", line 638, in _read_serial
    self.read_doc(docname, app)
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/environment.py", line 791, in read_doc
    pub.publish()
  File "/home/www/www/venv/lib/python3.4/site-packages/docutils/core.py", line 218, in publish
    self.apply_transforms()
  File "/home/www/www/venv/lib/python3.4/site-packages/docutils/core.py", line 199, in apply_transforms
    self.document.transformer.apply_transforms()
  File "/home/www/www/venv/lib/python3.4/site-packages/docutils/transforms/__init__.py", line 171, in apply_transforms
    transform.apply(**kwargs)
  File "/home/www/www/venv/lib/python3.4/site-packages/sphinx/transforms.py", line 332, in apply
    self.document.note_implicit_target(section_node)
  File "/home/www/www/venv/lib/python3.4/site-packages/docutils/nodes.py", line 1383, in note_implicit_target
    self.set_name_id_map(target, id, msgnode, explicit=None)
  File "/home/www/www/venv/lib/python3.4/site-packages/docutils/nodes.py", line 1330, in set_name_id_map
    self.set_duplicate_name_id(node, id, name, msgnode, explicit)
  File "/home/www/www/venv/lib/python3.4/site-packages/docutils/nodes.py", line 1367, in set_duplicate_name_id
    old_node = self.ids[old_id]
KeyError: 'philosophy'

[timgraham]

I dumped self.ids for this traceback:

Exception occurred:
  File "/home/tim/.virtualenvs/djangoproject-1.9/lib/python3.4/site-packages/docutils/nodes.py", line 1372, in set_duplicate_name_id
    old_node = self.ids[old_id]
KeyError: 'philosophy'

self.ids:

{'writing-more-views': <section "writing more views; écriture de vues supplémentaires": <title...> <paragraph...><snippet_with_filename...><parag ...>, 'wikipedia-s-entry': <target "wikipedia's entry": >, 'use-the-template-system': <section "use the template system": <title...><paragraph...><snippet_with_filename...><parag ...>, 'a-shortcut-render': <section "a shortcut: render(); un raccourci : render()": <title...><paragraph...><snippet_with_filename...><parag ...>, 'removing-hardcoded-urls-in-templates': <section "removing hardcoded urls in templates": <title...><paragraph...><literal_block...><paragraph...> ...>, 'write-your-first-view': <section "write your first view; écriture d'une première vue": <title...><paragraph...><snippet_with_filename...><parag ...>, 'namespacing-url-names': <section "namespacing url names": <title...><paragraph...><paragraph...><snippet_with_file ...>, 'a-shortcut-get-object-or-404': <section "a shortcut: get_object_or_404(); un raccourci : get_object_or_404()": <title...><paragraph...><snippet_with_filename...><parag ...>, 'write-views-that-actually-do-something': <section "write views that actually do something; écriture de vues qui font réellement des choses": <title...><paragraph...><paragraph...><paragraph...><par ...>, 'raising-a-404-error': <section "raising a 404 error; les erreurs 404": <title...><paragraph...><snippet_with_filename...><parag ...>, 'url-argument-kwargs': <section "url() argument: kwargs; paramètre d'url() : kwargs": <title...><paragraph...>>, 'url-argument-view': <section "url() argument: view; paramètre d'url() : view": <title...><paragraph...>>, 'writing-your-first-django-app-part-3': <section "writing your first django app, part 3; écriture de votre première application django, 3ème partie": <title...><paragraph...><section "philosophy; philosophi ...>, 'url-argument-regex': <section "url() argument: regex; paramètre d'url() : regex": <title...><paragraph...><paragraph...><paragraph...><par ...>, 'philosophie': <admonition "philosophie": <title...><paragraph...><paragraph...>>, 'url-argument-name': <section "url() argument: name; paramètre d'url() : name": <title...><paragraph...>>}

It looks like docutils is looking for the English version, 'philosophy', but the ids contain the translated version, 'philosophie'.

I haven't taken time to understand how the translation process works yet. Do you have any ideas before I proceed further?

[claudep]

I see there's now a 1.3.5 release of sphinx. Might worth a try?

[timgraham]

The issues persist with sphinx 1.3.5.

[claudep]

Is this KeyError systematic or random? I cannot replicate the error locally with Python 3.4 and Sphinx 1.3.5. I can build the French translation successfully.

[timgraham]

It always seems to occur with the 1.8 or 1.7 French docs (not 1.9). Change the queryset in update_docs.py to DocumentRelease.objects.filter(release_id='1.8', lang='fr')

[timgraham]

We'll try excluding versions of the translated documentation that is older than the current stable version from the build since these versions aren't updated anyway. They remain accessible but aren't updated.