Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: cleanup of old/deprecated documents #7994

Merged
merged 5 commits into from Mar 25, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/contribute.rst
Expand Up @@ -25,7 +25,7 @@ Contributing to development

If you want to deep dive and help out with development on Read the Docs, then
first get the project installed locally according to the
:doc:`installation guide </development/standards>`. After that is done we
:doc:`installation guide </development/install>`. After that is done we
suggest you have a look at tickets in our issue tracker that are labelled `Good
First Issue`_. These are meant to be a great way to get a smooth start and
won't put you in front of the most complex parts of the system.
Expand Down
2 changes: 1 addition & 1 deletion docs/development/front-end.rst
Expand Up @@ -55,7 +55,7 @@ in.
Getting Started
---------------

You will need to follow our :doc:`guide to install a development Read the Docs instance </development/standards>` first.
You will need to follow our :doc:`guide to install a development Read the Docs instance </development/install>` first.

The sources for our bundles are found in the per-application path
``static-src``, which has the same directory structure as ``static``. Files in
Expand Down
2 changes: 1 addition & 1 deletion docs/development/index.rst
Expand Up @@ -8,7 +8,7 @@ for development or taking the open source Read the Docs codebase for your own cu
:maxdepth: 1

architecture
standards
install
design
docs
front-end
Expand Down
136 changes: 74 additions & 62 deletions docs/development/standards.rst → docs/development/install.rst
@@ -1,76 +1,22 @@
Development Setup and Standards
===============================
Install local development instance
humitos marked this conversation as resolved.
Show resolved Hide resolved
==================================

.. meta::
:description lang=en: Install a local development instance of Read the Docs with our step by step guide.

These are development setup and standards that are adhered to by the core development team while
These are development setup and :ref:`standards <Core team standards>` that are adhered to by the core development team while
developing Read the Docs and related services. If you are a contributor to Read the Docs,
it might a be a good idea to follow these guidelines as well.

.. note::

Core team standards
-------------------

Core team members expect to have a development environment that closely
approximates our production environment, in order to spot bugs and logical
inconsistencies before they make their way to production.

This solution gives us many features that allows us to have an
environment closer to production:

Celery runs as a separate process
Avoids masking bugs that could be introduced by Celery tasks in a race conditions.

Celery runs multiple processes
We run celery with multiple worker processes to discover race conditions
between tasks.

Docker for builds
Docker is used for a build backend instead of the local host build backend.
There are a number of differences between the two execution methods in how
processes are executed, what is installed, and what can potentially leak
through and mask bugs -- for example, local SSH agent allowing code check
not normally possible.

Serve documentation under a subdomain
There are a number of resolution bugs and cross-domain behavior that can
only be caught by using `USE_SUBDOMAIN` setting.

PostgreSQL as a database
It is recommended that Postgres be used as the default database whenever
possible, as SQLite has issues with our Django version and we use Postgres
in production. Differences between Postgres and SQLite should be masked for
the most part however, as Django does abstract database procedures, and we
don't do any Postgres-specific operations yet.

Celery is isolated from database
Celery workers on our build servers do not have database access and need
to be written to use API access instead.

Use NGINX as web server
All the site is served via NGINX with the ability to change some configuration locally.

MinIO as Django storage backend
All static and media files are served using Minio --an emulator of S3,
which is the one used in production.

Serve documentation via El Proxito
Documentation is proxied by NGINX to El Proxito and proxied back to NGINX to be served finally.
El Proxito is a small application put in front of the documentation to serve files
from the Django Storage Backend.

Search enabled by default
Elasticsearch is properly configured and enabled by default.
All the documentation indexes are updated after a build is finished.
We do not recommend to follow this guide to deploy an instance of Read the Docs for production usage.
Take into account that this setup is only useful for developing purposes.


Set up your environment
-----------------------

After cloning ``readthedocs.org`` repository, you need to


#. install `Docker <https://www.docker.com/>`_ following `their installation guide <https://docs.docker.com/install/>`_.

#. clone the ``readthedocs.org`` repository:
Expand Down Expand Up @@ -112,7 +58,17 @@ After cloning ``readthedocs.org`` repository, you need to

inv docker.up --init # --init is only needed the first time

#. go to http://localhost:9000/ (MinIO S3 storage backend), click "..." and then "Edit Policy" and give "Read Only" access on all the buckets (``static`` and ``media``).
#. add read permissions to the storage backend:

* go to http://localhost:9000/ (MinIO S3 storage backend)
* login as admin / password
* click "..." next to the bucket name and then "Edit Policy"
* give "Read Only" access on all the buckets (``static`` and ``media``)

.. note::

``media`` bucket may be created after the first build is finished.
You will need to repeat this step after that.

#. go to http://community.dev.readthedocs.io to access your local instance of Read the Docs.

Expand All @@ -137,7 +93,7 @@ save some work while typing docker compose commands. This section explains these
``inv docker.shell``
Opens a shell in a container (web by default).

* ``--running`` the shell is open in a container that it's already running
* ``--no-running`` spins up a new container and open a shell
* ``--container`` specifies in which container the shell is open

``inv docker.manage {command}``
Expand Down Expand Up @@ -253,3 +209,59 @@ For others, the webhook will simply fail to connect when there are new commits t
* Take the "Client ID" and "Secret" for each service and enter it in your local Django admin at:
``http://community.dev.readthedocs.io/admin/socialaccount/socialapp/``.
Make sure to apply it to the "Site".


Core team standards
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify or remove this section as everything that is mentioned is done in docker already.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's worth noting this stuff in terms of architecture, but not necessarily in the install doc. I think it's fine tho.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll keep it here for now. However, I'd be 👍 on moving this into the https://docs.readthedocs.io/en/stable/development/architecture.html together with a nicer graph there.

-------------------

Core team members expect to have a development environment that closely
approximates our production environment, in order to spot bugs and logical
inconsistencies before they make their way to production.

This solution gives us many features that allows us to have an
environment closer to production:

Celery runs as a separate process
Avoids masking bugs that could be introduced by Celery tasks in a race conditions.

Celery runs multiple processes
We run celery with multiple worker processes to discover race conditions
between tasks.

Docker for builds
Docker is used for a build backend instead of the local host build backend.
There are a number of differences between the two execution methods in how
processes are executed, what is installed, and what can potentially leak
through and mask bugs -- for example, local SSH agent allowing code check
not normally possible.

Serve documentation under a subdomain
There are a number of resolution bugs and cross-domain behavior that can
only be caught by using `USE_SUBDOMAIN` setting.

PostgreSQL as a database
It is recommended that Postgres be used as the default database whenever
possible, as SQLite has issues with our Django version and we use Postgres
in production. Differences between Postgres and SQLite should be masked for
the most part however, as Django does abstract database procedures, and we
don't do any Postgres-specific operations yet.

Celery is isolated from database
Celery workers on our build servers do not have database access and need
to be written to use API access instead.

Use NGINX as web server
All the site is served via NGINX with the ability to change some configuration locally.

MinIO as Django storage backend
All static and media files are served using Minio --an emulator of S3,
which is the one used in production.

Serve documentation via El Proxito
Documentation is proxied by NGINX to El Proxito and proxied back to NGINX to be served finally.
El Proxito is a small application put in front of the documentation to serve files
from the Django Storage Backend.

Search enabled by default
Elasticsearch is properly configured and enabled by default.
All the documentation indexes are updated after a build is finished.
2 changes: 1 addition & 1 deletion docs/development/search.rst
Expand Up @@ -9,7 +9,7 @@ Currently we are using `Elasticsearch 6.3`_.
Local Development Configuration
-------------------------------

Elasticsearch is installed and run as part of the :doc:`development setup </development/standards>`.
Elasticsearch is installed and run as part of the :doc:`development setup </development/install>`.

Indexing into Elasticsearch
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
2 changes: 1 addition & 1 deletion docs/gsoc.rst
Expand Up @@ -33,7 +33,7 @@ the more likely we are to choose your application!
Getting Started
---------------

The :doc:`/development/standards` doc is probably the best place to get going.
The :doc:`/development/install` doc is probably the best place to get going.
It will walk you through getting a basic environment for Read the Docs setup.

Then you can look through our :doc:`/contribute` doc for information on how to get started contributing to RTD.
Expand Down
2 changes: 1 addition & 1 deletion docs/index.rst
Expand Up @@ -195,7 +195,7 @@ of Read the Docs and the larger software documentation ecosystem.

* **Getting involved with Read the Docs**:
:doc:`Contributing <contribute>` |
:doc:`Development setup </development/standards>` |
:doc:`Development setup </development/install>` |
humitos marked this conversation as resolved.
Show resolved Hide resolved
:doc:`roadmap` |
:doc:`Code of conduct <code-of-conduct>`

Expand Down