Skip to content

Loading…

Currently almost impossible to run a private instance without forking and making changes. #364

Closed
mrmachine opened this Issue · 4 comments

5 participants

@mrmachine

I only want to run a private instance because we have private GitHub and BitBucket repositories. I'm not stuck behind a VPN or firewall. If RTD allowed this, I would happily even pay RTD to host the docs for our private repos.

I think the easiest way to do this in the short term (at least for GitHub and BitBucket), is to change the project form validation to allow private repo URLs (git@github... or git@bitbucket...) and tell people to give read permission on their private repos to the RTFD GitHub or BitBucket accounts.

This would make it very easy for users to manage authentication and give RTFD access to their private repos through the GitHub or BitBucket site, without having to build in support for individual SSH keys and other authentication methods.

If this is not likely to happen any time soon, here are a few of the problems I encountered while trying to run a private instance of RTD:

1.

The use of relative imports requires hacking PYTHONPATH or sys.path (depending on your environment) so that packages inside the readthedocs package itself are importable.

This is ambiguous and it pollutes the global package namespace. You have imports like from core.forms import UserProfileForm. If anyone has another package or app called core (or projects, builds, api, bookmarks, etc.) then which one gets imported will depend on where in sys.path the readthedocs package is found.

This only works with the default manage.py because it is in the readthedocs package and python adds the current working directory to the start of sys.path. But when deploying under Apache2/mod_wsgi (for example), you won't know and can't rely on the CWD being anything in particular. You can use WSGIPythonHome to define your virtualenv root, but you'll have to manually hack sys.path in your wsgi.py or hack around with .pth files in your virtualenv.

These imports should be changed to from readthedocs.core... or those packages should be moved to the top level readthedocs.org folder, so they can be found and installed by find_packages() in setup.py.

2.

Even though I set USE_SUBDOMAIN = False, the SubdomainMiddleware still runs, and it looks like even if you wanted to use it, this middleware would never work for domains with more than 3 parts, e.g. rtd.mydomain.com.au is already 4 parts and project.rtd.mydomain.com.au is 5. Also "readthedocs.org" and "rtfd.org" are hard coded in here.

When I finally did try to move to production, I was getting 404s for every request, no matter the URL, but only when DEBUG = False. To work-around this, in my settings.py I need to do this:

from readthedocs.settings.base import *
MIDDLEWARE_CLASSES = list(MIDDLEWARE_CLASSES)
MIDDLEWARE_CLASSES.remove('core.middleware.SubdomainMiddleware')

3.

I also haven't used celeryd or redis before, so it was a bit of trial and error to find out that I needed to install and run redis when creating a project and even running manage.py update_repos myproject as shown in the docs didn't work.

Also the fact that sqlite settings were hard coded into manage.py and the layout of the settings package with settings, settings.base, settings.onebox, settings.postgresql and settings.sqlite required a bit of experimentation to find out which settings I actually needed to override (after extending settings.base) in my own settings file.

Just a little more guidance in the docs would help a lot here.

I'd be happy to try and work on these issues, if you have any feedback and direction or a road map on where and how you want to go with this stuff. I noticed that it making it easier to run private instances is on the TODO.

But I'd rather just pay you guys to host the docs for my private projects instead :)

Thanks for your time building RTD, and for reading this issue!

@danwiesenthal

Likewise, we would happily pay for hosting private docs of private repos.

@wraithan

I'll start by just mentioning that the private repos thing is completely off the table, and I have replied to that request before: #127

I will happily accept pull requests that make running private instances easier. Monolithic tickets like this are not actionable really. If you'd like to break it into multiple tickets, one for each problem you've encountered trying to run RTD privately, that'd be great.

1) We run under nginx/gunicorn in production, everything is geared around that. Which is why we allow for relative imports and such.

2) The middleware is very our production centric, I'd accept a pull request that disabled that if subdomains were disabled.

3) Can't really do much about your lack of experience with redis and celery, both being very very common in the django world. As is using DJANGO_SETTINGS_MODULE environment variable to switch what settings you are using when running in production. If you have additional docs that you'd like to add, I'd accept them.

tl;dr: hosting private repos wont happen and please break your issues out in the future.

Thanks for the feedback, sorry I missed this ticket when it was intiially opened. It was near the time that we switched primary maintainers.

@mgielda

@wraithan, how does this relate to the branch private_repos and this commit: 8d90460 ?

as far as I understand this some introductory work to get the feature but this idea has since been abandoned?

Anyway, @mrmachine it seems that the public service will not provide this feature at any time, which they understandably have a right not to do. But I guess there are many who are also interested to have this really, really great technology working in a private scope, like a gitlab instance+private rtfd server (or private github/bitbucket repos).

This is because not all projects (take e.g. customer projects) can be so easily opensourced, but by sharing a workflow between the open and closed source projects and benefitting from a continuous-docs setup one can actually make the case for the open source approach stronger ('let's do that open source, look, it's just one click away'), both inside and between organisations.

Also, it would just be so cool to have this kind of docs for everything we do, however the code is hosted.

I'd be very happy to offer some help if you took the time to open up separate tickets which would enable pulling from private repos and hosting the build docs on a local instance.

Cheers,
Michael

@ericholscher
Read the Docs member

Closing this because it isn't really actionable. Please open tickets (or better yet pull requests) addressing specific issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.