Update deployment docs with new staging branch info (#291)

* update contributing with info about staging branch

* add notes about staging branch

* formatting

* updating docs about staging branch and staginghub

* change to docs dir before building

* adding code formatting

CONTRIBUTING.rst

@@ -5,7 +5,6 @@ Contributing Guidelines
 This repository contains the code needed to maintain and update the Earth Lab
 Jupyter Hub.
 
-
 Get Started!
 ============
 
@@ -48,11 +47,20 @@ that describes the changes that you are making::
 
 Now you can make your changes locally.
 
+There are two protected branches in this repo - `master` and `staging`. Travis
+runs on both branches, and both branches are deployed to the same Google Cloud
+cluster.
+
+If you are proposing changes that affect deployment (e.g. changing `.travis.yml`
+or `deploy.py`) make and test those changes on the `staging` branch before
+creating a feature branch from `master`. See the `operations guide <https://earthlab-hub-ops.readthedocs.io/en/latest/daily-operations/index.html>`_ for details.
+
 5. Build the Docs Locally
 -------------------------
 
 Ensure that the tests pass, and the documentation builds successfully::
 
+    $ cd docs
     $ make html
 
 **Note to Windows users**

docs/source/daily-operations/deployment-workflow.rst

@@ -5,6 +5,20 @@ Deployment
 
 Deployment of the hubs is managed with continuous integration (TravisCI), rather than through use of the command-line kubernetes tools. Pull requests to the master branch trigger travis, as does merging a pull request. The Travis steps are scripted using python rather than calling :code:`kubectl` and :code:`helm` directly.
 
+There are two protected branches in this repo - `master` and `staging`. Travis
+runs on both branches, and both branches are deployed to the same Google Cloud
+cluster.
+
+* :code:`master` : This branch contains the production hubs - the ones we are
+  using for courses. We use this branch through a pull request workflow, and
+  deployment happens when a PR is merged.
+
+* :code:`staging` : This branch is for testing, especially for testing changes
+  to deployment (i.e. changes to :code:`.travis.yml` or :code:`deploy.py` that
+  go beyond
+  adding / removing hubs). You can safely test deployment changes here without
+  affecting production hubs. We commit / push directly to this branch, and it
+  never gets merged to master.
 
 User workflow
 -------------
@@ -15,12 +29,16 @@ When making changes to a hub (including both :ref:`new-hub` and :ref:`modify-rem
 * create a pull request with your changes - this will initiate a travis run that tests the builds of the docker image(s) and the docs. This does not push to Dockerhub, and does not deploy the hub. More specifically, this build `skips` the :code:`before_deploy` and :code:`deploy` sections in :code:`.travis.yml`.
 * after the PR is merged, travis builds the docker image(s), pushes them to dockerhub, and deploys the hub(s). This build `runs` the :code:`before_deploy` and :code:`deploy` sections in :code:`.travis.yml`.
 
-Before making changes to a production hub (i.e. one being used for a course), test your changes on :code:`staginghub` and only when that deploys correctly, copy these changes to the production hub.
+Before making changes to a production hub (i.e. one being used for a course),
+try them out on a test hub on :code:`staging` branch before creating a feature
+branch with the changes and pull requesting to master.
 
 Deploy script
 -------------
 
-The :code:`deploy.py` script at the top-level of this repo manages docker builds and deployments. The expectation is that this is only called by travis, not directly by the user (see the workflow, below).
+The :code:`deploy.py` script at the top-level of this repo manages docker builds and deployments. The expectation is that this is only called by travis, not directly by the user (see the workflow, below). Changes to the `deploy.py`
+script should be thoroughly tested on `staging` before creating a feature
+branch and PR to master.
 
 Basic usage is :code:`python deploy.py chartname`. The following options are available:
 
@@ -42,5 +60,4 @@ Then, for each deployments found with :code:`kubectl -n <chartname> get deployme
 
   kubectl rollout status --namespace <chartname> --watch <deployment>
 
-
 .. Warning:: If you do run :code:`deploy.py` locally, use the :code:`--no-setup` flag to avoid authenticating to gcloud as the travis user and borking your local access to the cluster. If that does happen, you can delete your :code:`.kube` directory and re-authenticate to gcloud.

docs/source/daily-operations/hub-monitoring.rst

@@ -37,8 +37,8 @@ state of affairs) run ``kubectl get pods --all-namespaces``. This will list all
 pods that are running, including service pods that we don't ever interact with.
 
 You should see at least two pods in each namespace that is associated to a hub.
-The namespace and hubname are the same, so ``staginghub`` lives in the
-``staginghub`` namespace.
+The namespace and hubname are the same, so an ``eahub`` hub lives in the
+``eahub`` namespace.
 
 Pods in the ``kube-system``, ``monitoring`` and ``router`` namespaces are best
 left alone.

docs/source/daily-operations/manage-storage.rst

@@ -77,7 +77,7 @@ When you edit the pvc, you will edit the size in the `spec` section of the yaml:
 
 Save your changes, and check the new limit::
 
-  kubectl describe pvc claim-kcranston -n staginghub
+  kubectl describe pvc claim-kcranston -n ea-hub
 
 You will see the message "Waiting for user to (re-)start a pod to finish file
 system resize of volume on node." An admin user can stop the user's pod with

docs/source/daily-operations/modify-remove-hub.rst

@@ -4,6 +4,10 @@
 Manage, Modify or Remove a Hub
 ===============================
 
+These instructions are for modifying a JupyterHub. If you want to understand
+more about how deployment works, or want to modify how we do deployment, see
+:doc:`deployment </daily-operations/deployment-workflow>`.
+
 Making Changes to an Existing Hub
 ---------------------------------
 
@@ -128,22 +132,24 @@ To check that your :code:`helm` command is properly configured run :code:`helm l
 This will list all helm releases that are currently installed. It should look
 similar to this::
 
-    NAME      	REVISION	UPDATED                 	STATUS  	CHART               	NAMESPACE
-    earthhub  	24      	Thu Jul 26 16:53:46 2018	DEPLOYED	earthhub-0.1.0      	earthhub
-    ingress   	2       	Tue Jul  3 18:09:46 2018	DEPLOYED	nginx-ingress-0.22.1	router
-    lego      	1       	Thu Jun 21 16:19:50 2018	DEPLOYED	kube-lego-0.4.2     	router
-    monitoring	28      	Thu Jul 26 16:54:03 2018	DEPLOYED	monitoring-0.1.0    	monitoring
-    staginghub	25      	Thu Jul 26 16:53:30 2018	DEPLOYED	staginghub-0.1.0    	staginghub
-    wshub     	18      	Thu Jul 26 16:54:11 2018	DEPLOYED	wshub-0.1.0         	wshub
+  NAME        	REVISION	UPDATED                 	STATUS  	CHART               	APP VERSION	NAMESPACE
+  cert-manager	2       	Wed Jun 17 10:36:47 2020	DEPLOYED	cert-manager-v0.15.1	v0.15.1    	cert-manager
+  ea-hub      	19      	Fri Sep 18 14:01:53 2020	DEPLOYED	earthhub-0.1.0      	           	ea-hub
+  edsc-hub    	2       	Wed Aug 26 21:26:46 2020	DEPLOYED	edsc-hub-0.1.0      	           	edsc-hub
+  ingress     	3       	Tue Jul 31 06:23:04 2018	DEPLOYED	nginx-ingress-0.23.0	0.15.0     	router
+  lego        	3       	Sun Oct 14 12:16:18 2018	DEPLOYED	kube-lego-0.4.2     	v0.1.6     	router
+  monitoring  	162     	Fri Sep 18 14:02:34 2020	DEPLOYED	monitoring-0.1.0    	           	monitoring
+  nbgrader-hub	7       	Fri Sep 18 14:00:24 2020	DEPLOYED	nbgrader-hub-0.1.0  	           	nbgrader-hub
+  staginghub  	63      	Tue Sep 29 13:38:40 2020	DEPLOYED	staginghub-0.1.0    	           	staginghub
 
 Depending on how many hubs are running there will be at least three releases
-deployed: :code:`ingress`, :code:`lego`, and :code:`monitoring`. These support
-all hubs and should never be removed. In the case shown above there are three
-hubs running: :code:`staginghub`, :code:`wshub` and :code:`earthhub`.
+deployed: :code:`ingress`, :code:`cert-manager`, and :code:`monitoring`. These support
+all hubs and should never be removed. In the case shown above there are four
+hubs running: :code:`ea-hub`, :code:`edsc-hub`, :code:`nbgrader-hub` and :code:`staginghub`.
 
-To delete the :code:`wshub` run::
+To delete the :code:`<hubname>` run::
 
-    helm delete wshub --purge
+    helm delete <hubname> --purge
 
 If you now
 visit :code:`https://hub.earthdatascience.org/<hubname>/` you should get a 404 error.
@@ -161,5 +167,5 @@ To permanently remove all storage (**THERE IS NO RECOVERING THE DATA AFTER DOING
 THIS!**) run the following command::
 
     kubectl delete namespace <hubname>
-    
+
 You have now deleted the hub and all of its storage.

docs/source/daily-operations/new-hub.rst

@@ -21,10 +21,7 @@ Z2JH helm chart. This raises two questions: which version should I use and how
 do I find out what versions are available?
 
 All versions of the JupyterHub helm charts are available from `<https://jupyterhub.github.io/helm-chart/>`_.
-We are currently using a `development release <https://jupyterhub.github.io/helm-chart/#development-releases-jupyterhub>`_
-of the chart for most hubs. The reason for this is that a lot of new features
-have been added but no new release has been made (should happen in August 2018).
-If you do not know better picking the latest development release is a good choice.
+We are generally use the latest stable release. The JupyterHub [heml chart changelog](https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/master/CHANGELOG.md) has all of the details about changes between versions.
 
 To change the version of the hub that you are using edit :code:`hub-charts/<hubname>/requirements.yaml`.
 The below snippet shows how to use :code:`v0.7-578b3a2`:
@@ -36,25 +33,23 @@ The below snippet shows how to use :code:`v0.7-578b3a2`:
       version: "v0.7-578b3a2"
       repository: "https://jupyterhub.github.io/helm-chart"
 
-You can also inspect what version :code:`hub-charts/staginghub/requirements.yaml` is
-using. Unless there are security related fixes or bugs that hinder your use of
-a specific version of a chart the recommendation is to not update your chart
+You can check `requirements.yml` file for other production hubs to see what version we are using elsewhere.
+
+Unless there are security related fixes or bugs that hinder your use of
+a specific version of a chart, we recommend not modifying the chart
 version during a workshop. Over the course of a semester it might be worth
 upgrading to the latest version, but should mostly be avoided.
 
-Take a look at :code:`staginghub/` as an example chart to base yours on. A chart can
-describe anything from a simple to a very complex setup. We typically use them
-for low complexity things. The most important file is :code:`values.yaml` which is
-where you configure your hub. Check the
-`zero to JupyterHub guide <http://zero-to-jupyterhub.readthedocs.io/>`_
-for ideas on what you might want to configure.
-
 Step one: Create a new hub directory
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 To begin your hub creation, first create a new directory in ``hub-charts/``
 with the name that you'd like your hub to have. The hub name should end with
-the word :code:`hub`.
+the word :code:`hub`. Then, it is simplest to copy over the
+:code:`Chart.yaml`, :code:`requirements.yaml`, and :code:`values.yaml` from
+another hub and edit them as you need. Check the
+`Zero to JupyterHub guide <http://zero-to-jupyterhub.readthedocs.io/>`_
+for ideas on what you might want to configure.
 
 You need to edit
 :code:`jupyterhub.hub.baseUrl` in your :code:`values.yaml` and set it to the same name
@@ -96,7 +91,8 @@ file:
 
     - |
       # Stage 3, Step XXX: Deploy the <HUBNAME>
-      python ./deploy.py --build --push --deploy <HUBNAME>
+      python ./deploy.py --build --push <HUBNAME>
+      python ./deploy.py --deploy <HUBNAME>
 
 Step four: Update the deploy.py file with the hub name
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -110,7 +106,7 @@ argument:
     argparser.add_argument(
         'chartname',
         help="Select which chart to deploy",
-        choices=['staginghub', 'earthhub', 'wshub', 'monitoring', '<HUBNAME>']
+        choices=['earthhub', 'wshub', 'monitoring', '<HUBNAME>']
     )
 
 Configuration values that need to remain secret can be stored in