Modification to support multi-GPU workstation access by users with varying level of access/resource allocation
Opinionated stacks of ready-to-run Jupyter applications in Docker.
If you're familiar with Docker, have it configured, and know exactly what you'd like to run, this one-liner should work in most cases:
docker run -d -P jupyter/<your desired stack>:<git-sha-tag>
If this is your first time using Docker or any of the Jupyter projects, do the following to get started.
- Install Docker on your host of choice.
- Open the README in one of the folders in this git repository.
- Follow the README for that stack.
Here's a diagram of the
FROM relationships between all of the images defined in this project:
Stacks, Tags, Versioning, and Progress
Starting with git commit SHA 9bd33dcc8688:
- Nearly every folder here on GitHub has an equivalent
jupyter/<stack name>on Docker Hub (e.g., all-spark-notebook → jupyter/all-spark-notebook).
latesttag in each Docker Hub repository tracks the
HEADreference on GitHub. This is a moving target and will make backward-incompatible changes regularly.
- Any 12-character image tag on Docker Hub refers to a git commit SHA here on GitHub. See the Docker build history wiki page for a table of build details.
- Stack contents (e.g., new library versions) will be updated upon request via PRs against this project.
- Users looking for reproducibility or stability should always refer to specific git SHA tagged images in their work, not
- For legacy reasons, there are two additional tags named
4.0on Docker Hub which point to images prior to our versioning scheme switch.
Other Tips and Known Issues
- If you haven't already, pin your image to a tag, e.g.
latestis a moving target which can change in backward-incompatible ways as packages and operating systems are updated.
- Python 2.x was removed from all images on August 10th, 2017, starting in tag
cc9feab481f7. If you wish to continue using Python 2.x, pin to tag
tini -- start-notebook.shis the default Docker entrypoint-plus-command in every notebook stack. If you plan to modify it in any way, be sure to check the Notebook Options section of your stack's README to understand the consequences.
- Every notebook stack is compatible with JupyterHub 0.5 or higher. When running with JupyterHub, you must override the Docker run command to point to the start-singleuser.sh script, which starts a single-user instance of the Notebook server. See each stack's README for instructions on running with JupyterHub.
- Check the Docker recipes wiki page attached to this project for information about extending and deploying the Docker images defined here. Add to the wiki if you have relevant information.
- The pyspark-notebook and all-spark-notebook stacks will fail to submit Spark jobs to a Mesos cluster when run on Mac OSX due to https://github.com/docker/for-mac/issues/68.
To build new images on Docker Cloud and publish them to the Docker Hub registry, do the following:
- Make sure Travis is green for a PR.
- Merge the PR.
- Monitor the Docker Cloud build status for each of the stacks, starting with jupyter/base-notebook and ending with jupyter/all-spark-notebook.
- See the stack hierarchy diagram for the current, complete build order.
- Manually click the retry button next to any build that fails to resume that build and any dependent builds.
- Avoid merging another PR to master until all outstanding builds complete.
- There's no way at present to propagate the git SHA to build through the Docker Cloud build trigger API. Every build trigger works off of master HEAD.
When there's a security fix in the Ubuntu base image, do the following in place of the last command:
ubuntu:16.04 SHA in the most-base images (e.g., base-notebook). Submit it as a regular PR and go through the build process. Expect the build to take a while to complete: every image layer will rebuild.
When there's a new stack definition, do the following before merging the PR with the new stack:
- Ensure the PR includes an update to the stack overview diagram in the top-level README.
- The source of the diagram is included in the alt-text of the image. Visit that URL to make edits.
- Ensure the PR updates the Makefile which is used to build the stacks in order on Travis CI.
- Create a new repoistory in the
jupyterorg on Docker Cloud named after the stack folder in the git repo.
- Grant the
stacksteam permission to write to the repo.
- Click Builds and then Configure Automated Builds for the repository.
jupyter/docker-stacksas the source repository.
- Choose Build on Docker Cloud's infrastructure using a Small node unless you have reason to believe a bigger host is required.
- Update the Build Context in the default build rule to be
- Toggle Autobuild to disabled unless the stack is a new root stack (e.g., like
- If the new stack depends on the build of another stack in the hierarchy:
- Hit Save and then click Configure Automated Builds.
- At the very bottom, add a build trigger named Stack hierarchy trigger.
- Copy the build trigger URL.
- Visit the parent repository Builds page and click Configure Automated Builds.
- Add the URL you copied to the NEXT_BUILD_TRIGGERS environment variable comma separated list of URLs, creating that environment variable if it does not already exist.
- Hit Save.
- If the new stack should trigger other dependent builds:
- Add an environment variable named NEXT_BUILD_TRIGGERS.
- Copy the build trigger URLs from the dependent builds into the NEXT_BUILD_TRIGGERS comma separated list of URLs.
- Hit Save.
- Adjust other NEXT_BUILD_TRIGGERS values as needed so that the build order matches that in the stack hierarchy diagram.
When there's a new maintainer, do the following:
- Visit https://cloud.docker.com/app/jupyter/team/stacks/users
- Add the new maintainer user name.
If automated builds have got you down, do the following:
- Clone this repository.
- Check out the git SHA you want to build and publish.
docker loginwith your Docker Hub/Cloud credentials.
make retry/release-all successfully pushes the last of its images to Docker Hub (currently
jupyter/all-spark-notebook), Docker Hub invokes the webhook which updates the Docker build history wiki page.