Skip to content

Commit

Permalink
Lock, build, and test commands
Browse files Browse the repository at this point in the history
Added `docker-compose.yml` and `Makefile` to help make it easier to lock, build, and test Python and R images.

Setting these up before we start having any big changes for '23, so we don't need to change our workflow.
  • Loading branch information
abkfenris committed Jun 21, 2023
1 parent 2b505d5 commit 2780d4c
Show file tree
Hide file tree
Showing 3 changed files with 69 additions and 3 deletions.
29 changes: 29 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
py-build: py-base/conda-linux-64.lock
docker compose stop py
docker compose build py

py-base/conda-linux-64.lock: py-base/environment.yml
conda-lock --kind explicit --platform linux-64 -f py-base/environment.yml
mv conda-linux-64.lock py-base/conda-linux-64.lock

py-lock:
conda-lock --kind explicit --platform linux-64 -f py-base/environment.yml
mv conda-linux-64.lock py-base/conda-linux-64.lock

py-lab:
docker compose up py

r-build: r/conda-linux-64.lock
docker compose stop r
docker compose build r

r/conda-linux-64.lock: r/environment.yml
conda-lock --kind explicit --platform linux-64 -f r/environment.yml
mv conda-linux-64.lock r/conda-linux-64.lock

r-lock:
conda-lock --kind explicit --platform linux-64 -f r/environment.yml
mv conda-linux-64.lock r/conda-linux-64.lock

r-lab:
docker compose up r
29 changes: 26 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,37 @@
# jupyter-image for OceanhackWeek

Repository for building OceanhackWeek 2022
Repository for building OceanhackWeek 2023
[JupyterHub](https://jupyter.org/hub) environments (Docker Images) with
[GitHub Actions CI](https://help.github.com/en/actions/automating-your-workflow-with-github-actions)

Docker images publicly [available here](https://github.com/orgs/oceanhackweek/packages?repo_name=jupyter-image). `jupyter-image` is the 2021 image, while `python` and `r` are for their matching profiles on the hub.

This repository contains configuration for the standard environment used during the OceanhackWeek 2021.
This repository contains configuration for the standard environment used during the OceanhackWeek 2023.
When you log into the OceanhackWeek JupyterHub you are running a virtual machine with Ubuntu 22.04,
a variety of command line tools like `vim` and `git`,
a `conda` Python/R environment with compatibly package versions,
and JupyterLab extensions such as `ipywidgets` and `ipyleaflet`.
By packaging everything up with Docker we help ensure that code written during the hackweek is reproducible and can be run on different physical hardware today and in the future.
By packaging everything up with Docker we help ensure that code written during the hackweek is reproducible and can be run on different physical hardware today and in the future.

## Development commands

There are a handful of helpful `make` commands for building and testing images.

- `py-lock` - Generate a new Python lockfile.
- `py-build` - Build Python Docker container (should regenerate lockfile if `environment.yml` was changed).
- `py-lab` - Launch JupyterLab for Python image. Watch the terminal output for a `127.0.0.1:8080/...` link with the access token.
- `r-lock` - Generate a new R lockfile.
- `r-build` - Build R Docker container (should regenerate lockfile if `environment.yml` was changed).
- `r-lab` - Launch JupyterLab for R image. Watch the terminal output for a `127.0.0.1:8080/...` link with the access token. Once in JupyterLab, you should be able to launch RStudio.

## Testing user-generated environments

Both images use [`nb_conda_kernels` ](https://github.com/Anaconda-Platform/nb_conda_kernels) which allows our users to create their own Conda environments.

This makes it so that we don't have to package everything into the images to start with.

To test that environments can be created, launch JupyterLab in one of the images.

- `conda create -n <env-name> <requirements>`, and make sure one of the requirements is a Jupyter kernel, like `ipykernel`.
- Try `conda activate <env-name>` and seeing if it's different than the base environment.
- The JupyerLab new notebook/console creation screen should now show options for `<env-name>`. This can sometimes take a few minutes before `nb_conda_kernels` sees it.
14 changes: 14 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
version: '3.4'

services:
py:
build: ./py-base
command: jupyter lab --port=8080 --ip=0.0.0.0
ports:
- "8080:8080"

r:
build: ./r
command: jupyter lab --port=8080 --ip=0.0.0.0
ports:
- "8080:8080"

0 comments on commit 2780d4c

Please sign in to comment.