Intermittent Build Failure due to CommonMark

[MikeDacre]

Details

• Project URL: https://readthedocs.org/projects/fyrd/
• Build URL (if applicable): https://readthedocs.org/projects/fyrd/builds/5785772/
• Read the Docs username (if applicable): mike.dacre

Expected Result

Build to work

Actual Result

About 80% of the time, I get the following error:

Running Sphinx v1.5.3
making output directory...

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/fyrd/envs/latest/lib/python3.5/site-packages/sphinx/application.py", line 512, in setup_extension
    mod = __import__(extension, None, None, ['setup'])
  File "/home/docs/checkouts/readthedocs.org/user_builds/fyrd/envs/latest/lib/python3.5/site-packages/sphinxarg/ext.py", line 13, in <module>
    from sphinxarg.markdown import parseMarkDownBlock
  File "/home/docs/checkouts/readthedocs.org/user_builds/fyrd/envs/latest/lib/python3.5/site-packages/sphinxarg/markdown.py", line 1, in <module>
    from CommonMark import Parser  # >= 0.5.6
ImportError: cannot import name 'Parser'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/fyrd/envs/latest/lib/python3.5/site-packages/sphinx/cmdline.py", line 295, in main
    opts.warningiserror, opts.tags, opts.verbosity, opts.jobs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/fyrd/envs/latest/lib/python3.5/site-packages/sphinx/application.py", line 199, in __init__
    self.setup_extension(extension)
  File "/home/docs/checkouts/readthedocs.org/user_builds/fyrd/envs/latest/lib/python3.5/site-packages/sphinx/application.py", line 516, in setup_extension
    err)
sphinx.errors.ExtensionError: Could not import extension sphinxarg.ext (exception: cannot import name 'Parser')

Extension error:
Could not import extension sphinxarg.ext (exception: cannot import name 'Parser')

The other 20% of the time, it works fine, with no changes to the code. I just have to fire off builds over and over again until one randomly works. It also works fine on my own computers.

[MikeDacre]

Now this happens every time, it is clearly caused by sphinx-argparse trying to use CommonMark and for some reason rtfd is installing CommonMark version 0.5.4, which is not working.

I have made this work for me by just removing sphinx-argparse and doing manual documentation, but that is hardly a solution.

[raamana]

I need this to be fixed also please.

[stefanhoelzl]

same problem here:
https://readthedocs.org/projects/mpy-dev-tools/builds/5988423/

[HannahVMeyer]

Are there any updates on this issue? I really like the arg-parse documentation, but am running into the same issues as described above: http://readthedocs.org/projects/limmbo/builds/6644960/

Are there any good alternatives for arg-parse?

Thanks

[vingar]

I'm hitting the same problem: https://readthedocs.org/projects/rucio/builds/6710110/

I tried to enforce CommonMark==0.7.4 in the project requirements and configured in rtds but it didn't help.

Would you know any work around ?

[MikeDacre]

This issue now has a workaround implemented in sphinx-argparse: alex-rudakov/sphinx-argparse#84.

The issue still isn't fixed though, there still should be a way to use CommonMark greater than 0.5.4 in rtfd builds.

[stsewd]

I'm not sure why is this happening, you should be able to download a new version of CommonMark by adding it to your requirements.txt file. Have anyone tried to wipe the environment before testing with a requirements file? (versions -> edit -> wipe)

[paulmueller]

I managed to get this working for some projects by editing requirements.txt:

sphinx-argparse==0.2.1
CommonMark==0.7.5

(see https://github.com/ZELLMECHANIK-DRESDEN/dclab/blob/master/docs/requirements.txt)

For other projects, this did not work (see #4855).

[humitos]

I read the whole thread and it's not clear to me what's the problem on Read the Docs. I mean, there are projects that are failing on their builds but it seems it's because of a problem in the external dependency sphinx-argparse.

What would be the action that we can take from Read the Docs, if any?

I was able to build properly in my dev instance without modifycation:

• mpy-dev-tools
• fyrd

limmbo failed because a conda package resolution problem:

ResolvePackageNotFound: 
  - libiconv==1.15=0
  - libxml2==2.9.8=0

and rucio failed with an issue that doesn't seems to be related to RTD:

Could not import extension rucio.common.doc.argparse.ext (exception: No module named rucio.common.doc.argparse.ext)

Please, let me know if there is any action that we can take from our side. Otherwise, I will close this issue.

[paulmueller]

@humitos I have not tested this on Python 2.7, but for Python 3 I would consider this problem resolved.

In general, the problem with sphinx-argparse in combination with RTD seems to be that detailed error messages are missing (see e.g. #4855 (comment)). Not sure who would be responsible for that.

[humitos]

the problem with sphinx-argparse in combination with RTD seems to be that detailed error messages are missing

Not sure to follow here. When you build locally in your computer do you have a better detailed message that explain the reasons (*)?

(*) those that you finally found with your custom extension, I think.

[paulmueller]

Yes, but I currently don't have a minimal example to support my claims. Give me some time and I will open a new issue.

[paulmueller]

I can reproduce the missing error messages locally, so this is not related to RTD.
alex-rudakov/sphinx-argparse#102

[humitos]

@paulmueller thanks for sharing your research here!

Now, I think that we can close this issue on RTD and people interested in this problem can subscribe to the issue that you opened in the sphinx extension.

Thank you all for debugging and helping us to make Read the Docs better ;)