docker-compose file for development

[humitos]

Idea of creating a docker-compose.yml file to allow new contributors.

(I'm not sure that this will work someday, but I wanted to leave the code in some place)

The steps would be:

git clone git@github.com:rtfd/readthedocs.org.git
cd readthedocs.org
docker build -f Dockerfile.devbuild -t readthedocs/build:dev --build-arg UID=1000 --build-arg GID=100 --build-arg GROUPNAME=users .
docker build -f Dockerfile.base -t readthedocs/build:docker-compose --build-arg UID=1000 --build-arg GID=100 --build-arg GROUPNAME=users --build-arg DOCKER_GID=991 .


docker-compose run --service-ports --user 1000 readthedocs

After that, you are ready to go to http://0.0.0.0:8000 and login as admin@example.com.

If non standar UID and GID are used, they should be changed.

At the moment I'm running into a permissions problem when the container wants to run

BuildEnvironmentWarning: Command python2.7 -mvirtualenv --no-site-packages --no-download /home/docs/readthedocs.org/user_builds/7/envs/latest failed:
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 2328, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 713, in main
    symlink=options.symlink)
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 925, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 1110, in install_python
    mkdir(lib_dir)
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 323, in mkdir
    os.makedirs(path)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/home/docs/readthedocs.org/user_builds/7/envs'

For some reason, the container that is created byt the RTD Python code mounts the shared volumes with root permissions.

(rtfd) [humitos@julia:readthedocs.org|master *+$%]$ docker exec -it build-12-project-34-6 bash
docs@build-12-project-34-6:~$ ls -la readthedocs.org/user_builds/
total 12
drwxr-xr-x 3 root root 4096 Nov 10 22:17 .
drwxr-xr-x 4 root root 4096 Nov 10 22:17 ..
drwxr-xr-x 2 root root 4096 Nov 10 22:17 6
docs@build-12-project-34-6:~$ python2.7 -mvirtualenv --no-site-packages --no-download /home/docs/readthedocs.org/user_builds/6/envs/latest
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 2328, in <module>
    main()
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 713, in main
    symlink=options.symlink)
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 925, in create_environment
    site_packages=site_packages, clear=clear, symlink=symlink))
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 1110, in install_python
    mkdir(lib_dir)
  File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 323, in mkdir
    os.makedirs(path)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/home/docs/readthedocs.org/user_builds/6/envs'

@safwanrahman maybe you find this interesting as a POC.

(the Dockerfiles could be very optimzed but since it's not working yet...)

[agjohnson]

There's actually a similar PR attempt open currently, #3003, but the additional problem that was never addressed there is how to handle updating the docker image for development purposes.

I'm less interested in a sanctioned dockerfile/docker compose file, as I won't use this or maintain this. But if the development workflow using docker compose could be addressed, that would be really handy for development.

Frankly, I'm not sure docker is required here. Why not just use supervisord and avoid confusion with Docker? The only dependency we require is redis in this case.

[maxirus]

I've written a Dockerfile and docker-compose.yml to bring RTD up in a microservice based fashion.

You can check it out on Docker Hub and my Git repo. I built it because I felt the barrier to entry was high when following the Local Install steps and I didn't want a monolithic service. Plus, it didn't really explain the different components and how they interact. Being Dockerized, it can be deployed virtually anywhere with one simple command and it's easy to see the different components. Further, it allows Enterprises to easily adopt RTD and hopefully contribute back as I am.

It's setup with an automated build pipeline between Git and Docker Hub. I'd like to contribute this back to the rtd project and have builds be triggered based on rtd's releases. Let me know what your thoughts are!

[agjohnson]

I'm going to close this PR, I'm not sure it's something we'd use or maintain at the moment.