Quickly and easily spin up a notebook server in GCP.
This repository includes utility functions for reading data from Google Cloud Storage.
If on a Mac or Windows, follow the instructions here to install Docker Toolbox.
In order to run a notebook or jobs remotely, get a service key in the GCS console. Once you've downloaded this key, rename it google_service_key.json and move it to the root directory of the repository.
See instructions here.
Run pip install --upgrade bumpversion
-
Make sure docker is running. Launch via
docker-machine -
Use the
deployment.shscript to orchestrate docker and run the notebook.
$ sh deployment.sh
Usage: sh deployment.sh [patch|build|push|deploy|port-forward|tear-down|run-local-notebook]-
To bump the version after making small changes, run
sh deployment.sh patch -
To build the docker image, run
sh deployment.sh build. -
To push the docker image to the remote container repository, run
sh deployment.sh push -
To deploy the notebook server, run
sh deployment.sh deploy. This will:- Make a new persistent disk (or use an existing one from a previous run)
- Spin up a Kubernetes cluster (or use an existing one from a previous run)
- Deploy a "pod" (server) to host the notebook on
-
To port-forward into a running pod, run
sh deployment.sh port-forward -
To tear down a running notebook cluster, run
sh deployment.sh tear-down -
To run the notebook locally, run
sh deployment.sh run-local-notebook. You never want to do this. Cause you can just run Jupyter in a virtualenv.NOTE Notebooks created on your local machine will not save automatically - they must be downloaded to persist.
To sync the contents of your persistent disk with your local machine (i.e. for versioning your notebooks and/or models), simply drop the script in this gist into the root directory of your repository.
sh sync.sh get <remote_path> <local_path>
sh sync.sh push <local_path> <remote_path>NOTE <remote_path> is always relative to "/home/jovyan/persistent_data"