Add support for automodule/autoclass etc. #26
Comments
|
So. I'm curious what you're suggesting that we do exactly. Currently, if you import your project straight away with autodoc, it won't work because that requires running arbitrary code. If we white list your user account, your conf.py files aren't cleaned up, and so everything that would normally work still works. So if you edit the sys.path in the conf.py, and don't have external dependencies, it should build fine. If we automagically added the project's directory to the sys.path, I feel we would be doing a disservice to the project maintainers, because they wouldn't make their documentation actually build correctly for everyone else. The only other option that I can think of (and you mention), is to create a virtualenv for each project, install the project into the venv, pulling in it's dependencies, and then building the documentation from the source package. This could probably be done, but it seems like a lot of work to support very few packages. Especially as we have added versions, and will add branch support in the future, that would be a good amount of processing power. I could probably be convinced to do this, but it seems like a lot of work to support few packages. |
|
Yes, I understand that it would be a lot of work, and potentially very expensive. Some smaller potential steps:
For me, the biggest bug was actually that the documentation appeared to build correctly, but various bits were missing, and there were no warnings before or after that this might be/was a problem. |
|
Hmm, apparently I 'closed' this bug. I don't know how I did that, or how to undo it - can't find any UI for that. I still think there are some action points that are worth considering, if the main one is two expensive. |
|
I pulled all the docs from your servers, and did some grepping: Of the 169 projects that actually have docs, 42 are using at least 1 autodoc directive (automodule, autoclass etc.) That's nearly 25%. There are 1774 autodoc directives overall - on average over 40 per project that uses autodoc. Almost all of these are going to be broken and missing some very large chunks of documentation. I think 25% is certainly worth worrying about, especially as there are no warnings, and nothing even in the docs themselves to say that something is missing. I would suggest at the very least:
Please don't hate me for reporting this! I just care about documentation, and docs that have been built incorrectly and are missing a lot is a big deal. As soon as I found this I deleted the docs for my project, because having an incomplete set of docs does my project a big misservice, and I can upload a correct set to packages.python.org. Here are the affected projects, BTW: booleanpy |
|
Documentation with autodoc works just fine. If the user is white listed (something we have to do on the backend, so that there is no arbitrary code execution). If they are correctly setting the sys.path in their conf.py, so that the documentation builds, it will work just fine. If there are autodoc warnings, they will indeed show up in the build output. I don't really know what you're talking about? http://readthedocs.org/builds/boothead/stargate/1445/ shows the autodoc errors because the user didn't set their sys.path correctly. http://sympy.readthedocs.org/modules/core.html is an example of autodoc working perfectly fine. I bet if you actually read the documentation on our site for most of these projects, I bet they are working fine. I agree that the whitelisting process could be more exposed (it's not really mentioned on the site). But allowing autodoc on any arbitrary repo seems like a really bad idea. There is a reason that we have done things this way. I'd be happy with suggestions (or patches) that improve this situation, other than exposing the limitations in the UI (which needs to be done ASAP). |
|
OK, sorry, we seem to be mis-communicating. I didn't know before what you meant by white-listing. There is nothing in the interface about this, so I had no idea what you meant - I thought you were speaking hypothetically. I did spot checks on some of the projects listed, and all the initial ones I found were like my own, where there big chunks of documentation missing but no errors shown in the documentation itself to say "you are missing something important here!". For example: http://readthedocs.org/docs/cli/latest/ (see API section at end) I have since discovered some that some projects build correctly. Regarding errors in the build standard output/error - sorry, they are there, I think I was confused by different errors - you get an error about the directive not being known - I now realise this is because you delete the conf.py unless the project is whitelisted. I think this is an important disclaimer to be added somewhere prominent. There are also some projects that include dependencies and still work e.g. Flask - which doesn't even set sys.path correctly: https://github.com/mitsuhiko/flask/blob/master/docs/conf.py And imports modules from outside standard lib: https://github.com/mitsuhiko/flask/blob/master/flask/app.py But is built right: |
|
I also realised why I thought the errors were not appearing on the build log output. On this page, for example: http://readthedocs.org/builds/lukaszb/vcs/427/ ... the text ** (ERROR/3) Unknown directive type "automodule". ** is hidden off the right hand side of the page, and it looks like debugging output, rather than an error. Sorry for the noise on that one. |
|
I went ahead and committed basic text that will alert people on the Import page. I also went through and whitelisted all the users in the system already, as nobody has tried anything funny, and am in the process of rebuilding all the docs (should take ~20 minutes). So at least the parts that we weren't informing users about will be fixed, if the docs are correct. Now new users will see the warning at the top of the import page (you won't, because your account is now whitelisted). I'd like to figure out a good way to not have to do this, but that will require more thinking. At least this will hopefully make it less invisible this will happen. I'll keep this ticket open because the problem isn't all the way fixed, but it should at least be better than before. Thanks for kicking my ass into gear. |
|
Hi ! From what I understood reading this thread, those errors that I got when trying to build my project (http://readthedocs.org/projects/spiteat/) : (ERROR/3) Unknown directive type "automodule". Come from the fact that I am not whitelisted ... and because of that, you generate a 'conf.py' file to build my project. Cheers, Sébastien |
|
Yea. This is because of the fact that autodoc executes arbitrary code, so anyone might be able to come onto the site and put a repo in that does something harmful. I've been looking into pypy's sandbox mode to see if I can take away this restriction, because it sucks a lot. I have whitelisted your user now, so you should be fine from here out. |
|
Ok great ! Yeah ... that's really a pity evilness is all around ! Because of those guys, we cannot even trust a poor autodoc. |
|
Hi, I know this is a closed ticket but I've encountered the same issue -- I'm user okfn on readthedocs. I've looked around the docs and the site but can't see how to request to get whitelisted. Am I missing something and please could you whitelist 'okfn'? BTW: ReadTheDocs is great! |
|
Whitelisted. Things should work now. If they don't check your build output On Mon, Aug 8, 2011 at 1:51 AM, rgrp <
Eric Holscher |
|
I have the same problem. Is there a standard procedure to be whitelisted? my username on readthedocs is 'gabrielelanaro' |
|
I'm having the same issue. could you please whitelist my user maraujop? |
|
I'm having the same issue. autodoc seems to be a pretty core aspect of sphrinx and hence something I was expecting to work. I'm not going to restructure my documentation to get around this. Can I be whitelisted please? gyllstromk |
|
Me too. Maybe we should implement some "please whitelist me" button? |
|
Can you please whitelist xaralis, we are using automodule in Ella's docs :( |
|
Done. On Wed, Feb 1, 2012 at 8:16 AM, xaralis <
Eric Holscher |
|
Hi, Can you please add me to the white list? My username is duanemalcolm. Cheers. |
|
Done. On Tue, Feb 7, 2012 at 3:15 PM, Duane Malcolm <
Eric Holscher |
|
Hey what about me? :-) |
|
Thanks Eric. Great idea, great website. |
|
Any updates on this ? I'm facing this issue right now. |
|
Forget, my mistake. |
|
can you please whitelist jfburkhart. |
|
I realize this issue is closed, but I am having problems with autodoc in my build. Has autodoc support been added to ReadTheDocs? I wasn't able to find anything explicitly in the documentation. My username on ReadTheDocs is swainn and the project I'm trying to build is here: https://bitbucket.org/swainn/gsshapy/src |
Currently, if docs use the directives automodule, autoclass etc supplied with extension sphinx.ext.autodoc, the docs are unlikely to build correctly.
The first reason is that the source code for the project is not on sys.path, and so autodoc cannot import it and find the docstrings/code that is needed to build the docs. However, even if you special case this module in your Sphinx conf.py, it still won't work if you have third-party dependencies that you import in those modules, because you will get an import error.
For instance, these docs are empty, but shouldn't be, presumably because 'import dulwich" fails on the build machine: http://readthedocs.org/docs/vcs/latest/api/backends/git.html
The following docs do appear, presumably because "import mercurial" happens to succeed on the build machine:
http://readthedocs.org/docs/vcs/latest/api/backends/hg.html
Interestingly, the warnings you would expect don't appear on the build page:
http://readthedocs.org/builds/lukaszb/vcs/427/
(I'm using someone else's package, just in case it was me being dumb, and because that package already does the sys.path hacking in conf.py to get it mainly working).
To build my docs locally, I have to activate the virtualenv that I use for developing my package. Perhaps we need something similar on readthedocs - like "mkvirtualenv $project.$version; setup.py develop". This could be quite an expensive operation though!
The text was updated successfully, but these errors were encountered: