This is a Docker configuration for running jupyter || lab with kernels for Haskell, julia, maxima (robert-dodier/maxima-jupyter), python, and R with support for multi-kernel notebooks via sos-notebook on Arch Linux. The name and framework are based on AustinRochford/notebooks.
See the Makefile for relevant commands.
If you have the Google Cloud SDK setup on your machine and are logged into an account with GCP GPU quota of 1 or greater, you should be able to run
make initialize_gcp_insecureand after about 15 minutes, access jupyter lab via HTTP at the external IP of the VM. However, you may want to investigate how to setup SSL/HTTPS access with initialize_gcp_secure.
You can stop with make stop_gcp and restart the insecure machine with make startup_gcp_insecure. Any data you save at the path $(DATA_DISK)/$(USER_NAME) should be (re)attached via a persistent disk named $(DATA_DISK) in your GCP account. Data you place anywhere else will be lost on stopping the container or VM. When you open jupyter lab, the file browser will be pointed at $(DATA_DISK)/$(USER_NAME)/$(NOTEBOOK_DIR).
You can destroy everything except your $(DATA_DISK) by running make delete_previous_gcp. This is controlled by GCP_VM_PREVIOUS=$(GCP_VM), which may need to be edited if changing other variables causes this value to fall out of sync with the state of your GCP account.
Images are available from Docker hub and GitHub Container Registry.
In addition to the jupyter kernels listed in the about section, the following software is installed by the Dockerfile
- arch linux packages listed in etc/pkglist-01.txt and etc/pkglist-02.txt
- yay AUR helper
- jupyter lab dependencies along with a number of jupyter lab extensions and their dependencies (not listed here as this has recently changed with the upgrade to jupyter lab
>=3.0). A default configuration is in etc/jupyter_notebook_config.py, which may be overridden by supplying container arguments in the cloud setup. - python packages listed in etc/python-libraries.txt
- julia packages listed in etc/Project.toml
- thoughtbot/rcm and thoughtbot/dotfiles
- zsh and ohmyzsh
- romkatv/powerlevel10k with default configuration in etc/p10k.zsh
- hlissner/doom-emacs
This section is deprecated. I use the containers generated by this project exclusively in a cloud computing environment. I currently only support GCP.
Interest in supporting cloud services other than GCP is welcome.
You will need to install the Google Cloud SDK. Ensure that you are logged in to an account associated with GCP.
$ gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
* <username of active account>
To set the active account, run:
$ gcloud config set account `ACCOUNT`You will of course require an active configuration and project.
$ gcloud config configurations describe <configuration>
is_active: true
name: <configuration>
properties:
compute:
region: us-central1
zone: us-central1-f
core:
account: <username of active account>
project: <project>
$ gcloud projects describe <project>
createTime: '2020-07-20T11:36:22.364Z'
lifecycleState: ACTIVE
name: notebooks
projectId: <project ID>
projectNumber: '<project number>'Once the Google Cloud SDK is configured, follow the list of Make targets that proceed from initialize_gcp_insecure in the Makefile. Insecure only refers to the lack of SSL (i.e. the notebook will be served over HTTP as opposed to HTTPS). To setup SSL, see initialize_gcp_secure and the section on Cloudflare. After running one of the initialize targets, machines can be stopped with the stop_gcp target and returned to a working state with the appropriate startup target such as startup_gcp_insecure.
If you would only like to implement password-based authentication, you will need to follow the instructions in the jupyter notebook documentation on preparing a hashed password. You can then edit the relevant line in the Makefile. The password in the git history (%HfuQRa@X%9&8MxM) should obviously be treated as compromised. You can generate the associated salted, hashed password in python
from jupyter_server.auth import passwd
passwd()
Enter password: %HfuQRa@X%9&8MxM
Verify password: %HfuQRa@X%9&8MxM
'sha1:0af606f6f6ce:11fe6ae47992d2d7a9015d322cf75e5a77c57149'The following is no longer needed. However, for reference, prior to jupyter server, this was done as follows with the notebook package:
from notebook.auth import passwd
passwd()
Enter password: %HfuQRa@X%9&8MxM
Verify password: %HfuQRa@X%9&8MxM
'argon2:$argon2id$v=19$m=10240,t=10,p=8$hQQSNsDLkgTth1v7IjN4Ig$G+O1EfHDdKq/hOZUODBnQA'If you are using an older version of jupyter you may need to do this. Note that the $ in this salted, hashed password had to be escaped as \$$.
\$$argon2id\$$v=19\$$m=10240,t=10,p=8\$$hQQSNsDLkgTth1v7IjN4Ig\$$G+O1EfHDdKq/hOZUODBnQAIt is assumed that data will be managed via a persistent disk named $DATA_DISK that will be attached in read-write mode to one running instance at a time. If you would like to run multiple instances of this container at the same time, you will need to account for the need to create multiple persistent disks.
This section is only relevant if you would like to access the jupyter notebook server at a custom domain via SSL. Cloudflare is used for managing Origin CA certificates. See the variables required by scripts/cloudflare-update.sh and checked by the Make target check_cf_env_set to setup the environment as necessary.
Once you have created the cf-cert.pem and cf-key.pem files, place them at ./etc/certs. The Make target ssl_cert_copy_to_gcp can then be used to copy them to the data persistent disk. So long as this disk is maintained and the certificates remain valid, this process should not need to be repeated. Of course, anyone who gains access to this disk will be able to read the contents of the certificates unless additional security measures are taken.
▶ tree -I 'maxima-jupyter'
.
├── Dockerfile
├── Dockerfile.dev
├── LICENSE
├── Makefile
├── README.md
├── VERSION
├── etc
│ ├── Project.toml
│ ├── certs
│ │ ├── cf-cert.pem
│ │ └── cf-key.pem
│ ├── gitconfig.local
│ ├── jupyter_notebook_config.py
│ ├── p10k.zsh
│ ├── pkglist-01.txt
│ ├── pkglist-02.txt
│ ├── pkglist-startup.txt
│ ├── pkglist-yay.txt
│ ├── plugin.jupyterlab-settings
│ ├── python-libraries.txt
│ ├── themes.jupyterlab-settings
│ ├── tracker.jupyterlab-settings
│ └── zshrc.local
├── notebooks
│ ├── ...
└── scripts
├── cloudflare-update.sh
├── install-cos-gpu.sh
└── startup.shThis code is distributed under the MIT License.