Grafana docs (#312)

* add docs for grafana setup

* change readme to reflect auto deployment

* separate commands for easier copying

docs/source/daily-operations/hub-monitoring.rst

@@ -18,6 +18,7 @@ a hub and observe what it looks like on the monitoring. Next time a workshop
 is happening keep an eye on the monitoring to see what happens when lots of
 people login at the same time.
 
+For more about the our Grafana setup, see :ref:`grafana_and_prometheus`.
 
 Operations
 ----------

docs/source/daily-operations/troubleshooting.rst

@@ -50,26 +50,40 @@ Helpful CLI commands
 
 You can view and edit components of your cluster from the command line using a combination of commands from Google Cloud SDK, Kubernetes, and Helm.
 
-GCloud commands::
+**GCloud commands**
 
-  $ gcloud containers cluster list  # list your clusters (probably only one; jhub)
+List your clusters (probably only one, `jhub`)::
 
-  $ gcloud container clusters describe jhub  # show details about the cluster ``jhub``
+  $ gcloud containers cluster list
 
-  $ gcloud auth list # list all gcloud authentications; highlights active authentication
+Show details about cluster jhub::
+
+  $ gcloud container clusters describe jhub
+
+List all gcloud authentications; highlights active authentication::
+
+  $ gcloud auth list
 
 **Kubernetes commands**
 
-You can view and edit your kubernetes cluster using `kubectl`. There is a great cheat sheet in the official docs: https://kubernetes.io/docs/reference/kubectl/cheatsheet/
+You can view and edit your kubernetes cluster using `kubectl`. There is a great cheat sheet in the official docs: https://kubernetes.io/docs/reference/kubectl/cheatsheet/ but here are some common ones.
+
+List namespaces::
+
+  $ kubectl get namespace
+
+List all pods in a namespace::
+
+  $ kubectl get pods -n <namespace>
 
-Short list of kubectl commands::
+Get logs for a single pod::
 
-  $ kubectl get namespace  # list of namespaces
+  $ kubectl logs -n <namespace> <podname>
 
-  $ kubectl get pods -n <namespace>  # list all pods in a namespace
+Get recent events for a pod::
 
-  $ kubectl logs -n <namespace> <podname> # get logs for a single pod
+  $ kubectl get events -n <namespace> --sort-by='{.lastTimestamp}'
 
-  $ kubectl get events -n <namespace> --sort-by='{.lastTimestamp}'  # get events for a pod
+Get list of PersistentVolumeClaims::
 
-  $ kubectl get pvc -n <namespace>  # get list of PersistentVolumeClaims
+  $ kubectl get pvc -n <namespace>

docs/source/setup/grafana.rst

@@ -0,0 +1,28 @@
+.. _grafana_and_prometheus:
+
+Grafana and Prometheus
+======================
+
+The `monitoring <https://github.com/earthlab/hub-ops/tree/master/monitoring>`_ hub implements a graphical display of hub resource usage at  `<https://grafana.hub.earthdatascience.org/>`_. This uses the open source projects `Grafana <https://grafana.com/>`_  and `Prometheus <https://prometheus.io/>`_. Prometheus collects, stores, and provides time series data about the hub operations. Grafana plots the data.
+
+We include helm charts for Grafana and Prometheus, and then there are additional settings (resource, etc) in values.yml. There are additional settings available through the Grafana web interface.
+
+Editing the Grafana hub list
+----------------------------
+
+When you add or remove hubs, the Grafana pages does not automatically update with the new hub list (there is probably a way to do this, but that's a project for another day). To modify the list of hubs available, you modify the settings in the web interface.
+
+To log in to Grafana from the `monitoring page <https://github.com/earthlab/hub-ops/tree/master/monitoring>`_, click the little arrow + box icon in the bottom right corner of the screen:
+
+  .. image:: ../media/login-icon.png
+        :width: 200
+
+The username is ``admin`` and the password is in ``secrets/monitoring.yml``. You can also retreive secrets using ``kubectl``. Username::
+
+   $ kubectl get secrets -n monitoring monitoring-grafana -o jsonpath='{.data.admin\-user}' | base64 -D
+
+and password::
+
+   $ kubectl get secrets -n monitoring monitoring-grafana -o jsonpath='{.data.admin\-password}' | base64 -D
+
+Once logged in, click on the settings (gear) icon at the top of the page. Go to the `Variables` tab, and modify the list for `$hubname`. Save your changes.

docs/source/setup/index.rst

@@ -23,3 +23,4 @@ Get started
    setup-helm
    git-crypt
    tooling
+   grafana

monitoring/README.md

@@ -1,8 +1,10 @@
 # Monitoring for the hubs
 
-Deploy from the top level directory with:
+These files define the `monitoring` hub that underlies https://grafana.hub.earthdatascience.org.
+
+Deployment of the monitoring hub happens automatically through travis and the deploy.py script. To deploy manually: 
+
 ```
 $ (cd monitoring && helm dep up)
 $ helm upgrade --install --namespace monitoring monitoring monitoring --version=v0.1.0 -f secrets/monitoring.yaml
 ```
-Needs to be added to auto deployment setup.