PXP-8779 HP-347 Feat/ecs

README.md

@@ -2,21 +2,23 @@
 
 Hatchery creates Kubernetes Pods for workspace services. Workspace services must expose HTTP servers. Ambassador is used to proxy user traffic through to their container workspace once it is launched by Hatchery.
 
-## Documentation TOC
-
-See [this bLog](https://www.divio.com/blog/documentation/) for an introduction to the different types of documentation (explanation, how-to, tutorial, reference).
+## Documentation
 
 ### Explanation
-* [hatchery overview](doc/explanation/hatcheryOverview.md)
-* [api](doc/explanation/hatcheryApi.md)
-* [dockstore apps](doc/explanation/dockstore.md)
+* [Hatchery overview](doc/explanation/hatcheryOverview.md)
+* [API documentation](doc/explanation/hatcheryApi.md)
+* [Configuring Dockstore apps](doc/explanation/dockstore.md)
 
 ### How-to
-* [dev-test](doc/howto/devTest.md)
-* [gen3-fuse](doc/howto/fuseSidecar.md)
-* [Jupyter Notebook](doc/howto/jupyterNotebook.md)
-* [R Studio](doc/howto/rStudio.md)
-* [Galaxy](doc/howto/galaxy.md)
+* [Hatchery configuration](doc/howto/configuration.md)
+* [Quick start for local development](doc/howto/quick_start.md)
+* [Run the tests locally](doc/howto/devTest.md)
+* [Configuring gen3-fuse](doc/howto/fuseSidecar.md)
+* Configuring workspaces:
+    * [Jupyter](doc/howto/jupyterNotebook.md)
+    * [RStudio](doc/howto/rStudio.md)
+    * [Galaxy](doc/howto/galaxy.md)
+    * [noNVC Firefox](doc/howto/noVNCFirefox.md)
 
 ### Tutorials
 

doc/howto/configuration.md

@@ -35,6 +35,7 @@ An example manifest entry may look like
       "user-uid": 1000,
       "fs-gid": 100,
       "user-volume-location": "/home/jovyan/pd",
+      "gen3-volume-location": "/home/jovyan/.gen3"
       "friends": []
     }]
   }
@@ -66,7 +67,8 @@ An example manifest entry may look like
     * `ready-probe` the path to use for the Kubernetes readiness probe.
     * `user-uid` the UID for the user in this container.
     * `fs-gid` the GID for the filesystem mounts.
-    * `user-volume-location` the location where the user persistant storage should be mounted in this container.
+    * `user-volume-location` the location where the user persistent storage should be mounted in this container.
+    * `gen3-volume-location` the location where the user's API key file will be put into
     * `lifecycle-pre-stop` a string array as the container prestop command.
     * `lifecycle-post-start` a string array as the container poststart command.
     * `friends` is a list of kubernetes containers to deploy alongside the main container and the sidecar in the kubernetes pod

doc/howto/jupyterNotebook.md

@@ -40,6 +40,7 @@ Jupyter lab is the successor to jupyter notebook.
       "lifecycle-post-start": ["/bin/sh","-c","export IAM=`whoami`; rm -rf /home/$IAM/pd/dockerHome; ln -s $(pwd) /home/$IAM/pd/dockerHome; mkdir -p /home/$IAM/.jupyter/custom; echo \"define(['base/js/namespace'], function(Jupyter){Jupyter._target = '_self';})\" >/home/$IAM/.jupyter/custom/custom.js; ln -s /data /home/$IAM/pd/; true"],
       "user-uid": 1000,
       "fs-gid": 100,
-      "user-volume-location": "/home/jovyan/pd"
+      "user-volume-location": "/home/jovyan/pd",
+      "gen3-volume-location": "/home/jovyan/.gen3"
     },
   ```

doc/howto/noVNCFirefox.md

@@ -63,6 +63,7 @@ In the main container's manifest, specify
     "export IAM=`whoami`; rm -rf /home/$IAM/pd/dockerHome; ln -s $(pwd) /home/$IAM/pd/dockerHome; mkdir -p /home/$IAM/.jupyter/custom; echo \"define(['base/js/namespace'], function(Jupyter){Jupyter._target = '_self';})\" >/home/$IAM/.jupyter/custom/custom.js; ln -s /data /home/$IAM/pd/; true"
   ],
   "user-volume-location": "/home/jovyan/pd",
+  "gen3-volume-location": "/home/jovyan/.gen3"
   "use-shared-memory": "true",
   "friends": [
     {

doc/howto/quick_start.md

@@ -0,0 +1,19 @@
+# Quick start for local development
+
+- Create a configuration file at `./hatchery.json` with basic configuration:
+
+```
+{
+    "user-namespace": "jupyter-pods",
+    "sub-dir": "/lw-workspace",
+    "user-volume-size": "10Gi"
+}
+```
+
+- Install [nodemon](https://nodemon.io/)
+
+- Run Hatchery:
+
+`export GEN3_ENDPOINT=qa-heal.planx-pla.net; export GEN3_VPCID=qaplanetv1; nodemon --exec go run main.go -config ./hatchery.json --signal SIGTERM`
+
+The API is exposed at http://0.0.0.0:8000.

doc/howto/rStudio.md

@@ -27,6 +27,7 @@ to disable authentication as this is a single user container when run with Hatch
       "use-tls": "false",
       "ready-probe": "/",
       "user-volume-location": "/home/rstudio/pd",
+      "gen3-volume-location": "/home/jovyan/.gen3"
       "fs-gid": 100
     }
 ```

go.mod

@@ -6,13 +6,18 @@ require (
 	github.com/DataDog/datadog-go v4.8.1+incompatible // indirect
 	github.com/DataDog/sketches-go v1.1.0 // indirect
 	github.com/Microsoft/go-winio v0.5.0 // indirect
+	github.com/apparentlymart/go-cidr v1.1.0
+	github.com/aws/aws-sdk-go v1.40.15
 	github.com/go-logr/logr v1.0.0 // indirect
+	github.com/gofrs/flock v0.8.1 // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
 	github.com/google/go-cmp v0.5.6 // indirect
+	github.com/google/pprof v0.0.0-20210804190019-f964ff605595 // indirect
 	github.com/google/uuid v1.3.0 // indirect
 	github.com/googleapis/gnostic v0.5.5 // indirect
 	github.com/json-iterator/go v1.1.11 // indirect
 	github.com/opentracing/opentracing-go v1.2.0 // indirect
+	github.com/sirupsen/logrus v1.8.1 // indirect
 	github.com/tinylib/msgp v1.1.6 // indirect
 	golang.org/x/net v0.0.0-20210726213435-c6fcb2dbf985 // indirect
 	golang.org/x/oauth2 v0.0.0-20210628180205-a41e5a781914 // indirect
@@ -29,6 +34,7 @@ require (
 	k8s.io/client-go v1.5.2
 	k8s.io/klog/v2 v2.10.0 // indirect
 	k8s.io/utils v0.0.0-20210802155522-efc7438f0176 // indirect
+	sigs.k8s.io/aws-iam-authenticator v0.5.3
 )
 
 replace (