diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 72d4fc4..39deec6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,7 +2,7 @@ Most of this repository is structured for **Sphinx**, a documentation engine in Python. -## Build the documentation +## Build the documentation locally The easiest way to build the documentation in this repository is to use `nox`, a tool for quickly building environments and running commands within them. This ensures that your environment has all the dependencies needed to build the documentation. @@ -27,3 +27,55 @@ To build live documentation that updates when you update local files, run the fo ``` nox -s docs_live ``` + +## Pre-populate the FreshDesk Help widget + +We use a [FreshDesk Help Widget](https://support.freshdesk.com/support/solutions/articles/239273-set-up-your-help-widget) to let users quickly submit support tickets from our documentation. +This is the little widget in the bottom-right that you can click on. + +Configuration for the JavaScript that loads this widget is in our `conf.py` file. +See the `widget_embed_code` section. + +### Create a button to make the widget pop up + +Trigger the Help widget by calling the `openWidget()` function in JavaScript. +You can attach it to a button by embedding some raw HTML in MyST Markdown like so: + +```html + +``` + + + +### Pre-fill parts of the support widget + +You can configure the button to pre-populate fields for the user so that they can ask make their request more quickly. +To do so, use `key:"value"` pairs in the call to `openWidget()`. + +See the `widget_embed_code` section of `conf.py` for the list of possible field names. + +The example below populates a few extra fields: + +```html + +```` + + + +:::{admonition} Things to pay attention to: + +- **Note curly brackets!** You need to give the `key:val` pairs inside curly brackets! +- **Type must exactly match a dropdown option**. If you get the case, spacing, etc wrong, then no option will be selected. +::: + +## Review and merging guidelines + +See the [2i2c team guidelines for reviewing and merging](https://infrastructure.2i2c.org/contributing/code-review/#review-and-merge-guidelines-for-2i2c-engineers). Use your best judgment! diff --git a/_static/custom.css b/_static/custom.css index 07b493f..f3b12bf 100644 --- a/_static/custom.css +++ b/_static/custom.css @@ -1,4 +1,19 @@ /* Tables */ table.align-middle td { vertical-align: middle !important; +} + +/* Default button style so we don't have to add classes each time */ +button { + border-radius: .5em; + padding: 1em; + background: var(--pst-color-primary); + color: white; + font-weight: bold; + border: none; +} + +/* This just makes the widget background a little bit nicer on dark mode */ +#launcher-frame { + border-radius: .5em; } \ No newline at end of file diff --git a/admin/howto/environment/index.md b/admin/howto/environment/index.md index 23e5ced..7d80966 100644 --- a/admin/howto/environment/index.md +++ b/admin/howto/environment/index.md @@ -110,6 +110,19 @@ research organizations use this. If you are using the default environment, and think that one or two packages should be installed by default on it, please [send a support request](../../../support.md) and request an update to the default environment. +(image-building-setup)= +## Allow users to build their own environment image + +Hubs can be configured to allow users to [dynamically build environment images from the JupyterHub](#user:environment-building). This provides a lightweight UI allowing users to point the hub to a GitHub repository containing the definition of the environment they wish to build. + + + +:::{admonition} Ask a 2i2c staff member to enable dynamic image building +This feature requires a bit of set-up to enable for a community's hub, so it isn't enabled by default. If you'd like this functionality on your hub, reach out to a 2i2c team member. +::: + ## Accessing user interfaces There are three main interfaces available on the 2i2c JupyterHubs. diff --git a/admin/howto/environment/update-community-image.md b/admin/howto/environment/update-community-image.md index 895c3d8..d308c54 100644 --- a/admin/howto/environment/update-community-image.md +++ b/admin/howto/environment/update-community-image.md @@ -175,9 +175,9 @@ Once complete, under the section *Repository secrets* you should now see two row //: ``` - e.g. `quay.io/jnywong/jupyter-scipy-xarray:739fec9705b1`, which you need to provide in the [](/user/howto/specify-unlisted-image.md#the-bring-your-own-image-option). + e.g. `quay.io/jnywong/jupyter-scipy-xarray:739fec9705b1`, which you need to provide in the [](#environment:bring-your-own-image). -1. Navigate to your 2i2c hub and paste the image tag into the *Image > Custom Image > Other...* field (see [](/user/howto/specify-unlisted-image.md#the-bring-your-own-image-option)). +1. Navigate to your 2i2c hub and paste the image tag into the *Image > Custom Image > Other...* field (see [](#environment:bring-your-own-image)). 1. Click start to launch the server and test your custom environment. You can continue editing the *DockerFile* and *environment.yml*, then push changes to the pull request as required. diff --git a/conf.py b/conf.py index d30bbf8..fe671af 100644 --- a/conf.py +++ b/conf.py @@ -81,7 +81,28 @@ # Embed a contact form Freshdesk widget to collect support tickets with widget_embed_code = """ // This is the function to open the widget code -function openWidget() { +function openWidget({ + name = '', + email = '', + subject = '', + description = '', + hub_url = '', + type = '' +} = {}) { + // Build prefill data with provided values + const prefillData = { + name: name, + email: email, + subject: subject, + description: description, + type: type, + cf_hub_url: hub_url, + }; + console.log(prefillData) + // Only prefill if any values were provided + if (name || email || subject || description || hub_url || type) { + FreshworksWidget('prefill', 'ticketForm', prefillData); + } FreshworksWidget('open'); } diff --git a/contribute/index.md b/contribute/index.md new file mode 100644 index 0000000..7a5a514 --- /dev/null +++ b/contribute/index.md @@ -0,0 +1,5 @@ +# Contributing guide + +```{include} ../CONTRIBUTING.md +:start-line: 2 +``` \ No newline at end of file diff --git a/index.md b/index.md index d593881..595e6ac 100644 --- a/index.md +++ b/index.md @@ -37,9 +37,8 @@ Covers end-user workflows that are common for cloud-native workflows with intera user/topics/getting-started user/topics/data/index +user/environment/index community/content -user/howto/specify-unlisted-image -user/topics/dynamic-imagebuilding/index user/howto/launch-dask-gateway-cluster ``` @@ -99,4 +98,5 @@ Lists and programmatically-generated content to serve as a quick reference. :maxdepth: 2 about/terminology +contribute/index ``` diff --git a/support.md b/support.md index a670001..b449735 100644 --- a/support.md +++ b/support.md @@ -9,7 +9,7 @@ When you make a support request, please include as much information as possible % Copy the style classes of sphinx-design buttons % The JavaScript that this calls is defined in conf.py - diff --git a/user/howto/specify-unlisted-image.md b/user/environment/choose.md similarity index 85% rename from user/howto/specify-unlisted-image.md rename to user/environment/choose.md index 9efe14f..93468f0 100644 --- a/user/howto/specify-unlisted-image.md +++ b/user/environment/choose.md @@ -1,7 +1,7 @@ (unlisted-image)= -# Customize the software and hardware for user environments +# Choose an environment for your JupyterHub server -To start a server on the Hub, you need to go to the hub's home page, then click on the `Start My server button`. +To start a server on the Hub, go to the hub's home page, then click on the `Start My server button`. Depending on how your hub was set up, after clicking the button, you will be either: @@ -28,6 +28,7 @@ The image above represents a `Server Options` page example, where you can choose - an image **profile** that defined the software environment and is curated by your hub administrator, with options such as _Python_, _R_, or even _Bring your own image_ - for each profile, you can select from a list of pre-configured **options** for the `Node share`, which represents the hardware resources such as CPU and RAM available on the server. +(environment:bring-your-own-image)= ### The "Bring your own image" option If this feature is enabled for your hub, then the user can freely specify any image hosted on a public container registry. @@ -49,3 +50,9 @@ We recommend always explicitly specifying a version number in the `TAG` field ra ```shell jovyan@user:~$ echo $JUPYTER_IMAGE ``` + +### The "Dynamic Image Building" option + +This allows users to dynamically build an environment image for their Jupyter server, similar to [mybinder.org](https://mybinder.org). + +See [](./dynamic-imagebuilding.md) for more information. diff --git a/user/environment/dynamic-imagebuilding.md b/user/environment/dynamic-imagebuilding.md new file mode 100644 index 0000000..ebfc390 --- /dev/null +++ b/user/environment/dynamic-imagebuilding.md @@ -0,0 +1,106 @@ +(user:environment-building)= +# Build your own environment from the hub + +Dynamic image building allows users to build their own environments on-the-fly, launch interactive sessions with those environments, and share built environments with other users. +It integrates the capabilities of [BinderHub](https://binderhub.readthedocs.io/en/latest/) with a JupyterHub. This flexibility enables users to manage their own software image environments in the cloud.[^blog] + +[^blog]: See [this blog post](https://2i2c.org/blog/2024/jupyterhub-binderhub-gesis/) for more details. + +:::{admonition} For hub administrators +:class: seealso +To enable dynamic image building, hub administrators should see [](#image-building-setup). +::: + +In the demo below, a user selects the {guilabel}`Build your own image` option from the launch dropdown, provides the URL of a GitHub repository containing [repo2docker configuration files](https://repo2docker.readthedocs.io/en/latest/configuration/), builds an environment image from this repository, and launches an interactive session with it. + + + +:::{warning} +Currently, user-built environments support only **GitHub** as a repository provider. +::: + +## Benefits + +* **No Dockerfile expertise needed** — `repo2docker` generates a Dockerfile automatically from commonly used configuration files of popular package managers, but allows advanced builds for those who are comfortable with Dockerfiles. +* **Reproducible** – your environment configuration is fully described in version-controlled files. +* **Portable to Binder** – the same setup works on [mybinder.org](https://mybinder.org) and any BinderHub, as well as other JupyterHubs with dynamic image building. + +## How dynamic image building works + +This JupyterHub configuration uses the [repo2docker](https://repo2docker.readthedocs.io/en/latest/) tool for building images in a way that JupyterHub can use. It was designed for the [BinderHub technology](https://binderhub.readthedocs.io) which powers [mybinder.org](https://mybinder.org). + +## Get started + +This is a quick guide to getting started with dynamic image building. + +### Step 1: Prepare a GitHub Repository + +1. Create a **public GitHub repository**. +2. Add one or more of the following optional files, based upon your needs: + + * `environment.yml` – for `conda` packages (and/or `pip` Python packages). + * `requirements.txt` – for `pip` Python packages. + * `install.R` – for R packages. + * `apt.txt` – for Ubuntu system packages. + * `runtime.txt` – to pin Python/R versions if not specified elsewhere. + * `postBuild` (executable) – script to run after installation. + * `start` (executable) – script to run each time the container starts. + + `repo2docker` dynamically generates a `Dockerfile` for you, based upon which of these files you include in your repo, so you don't have to deal with the intricacies of writing your own `Dockerfile`. For further information about these files, see the [repo2docker configuration file guide](https://repo2docker.readthedocs.io/en/latest/configuration/). + + Alternatively, if your repo includes a `Dockerfile`, then `repo2docker` uses it directly (ignoring any of the files above) and performs a regular Docker build. + + See [](#image-building:learn-more) for links to additional resources. + +### Step 2: Configure the JupyterHub “Build your own image” Option + +1. In JupyterHub, choose {bdg-primary}`Build your own image` from the {bdg-primary}`Image` dropdown in the launcher. +2. Enter your GitHub repo name (e.g. [binder-examples/requirements](https://github.com/binder-examples/requirements)) +3. Optionally include a branch, tag or commit to use. By default, the `HEAD` will use the default branch. +4. Click the {bdg-primary}`Build` button to start the build process. You can view the build logs to monitor progress. +5. Once the build is complete, choose the {bdg-primary}`Resource Allocation` (CPU, RAM) for the server launch. +6. Click {bdg-primary}`Start` to launch your server. + +### Step 3: Use and Share Your Environment + +1. If you re-launch the same repo and **reuse the same image later**, the build is cached by `repo2docker` unless you update your repo. +1. You can also **share your repo link** with others so they can build the same image in another dynamic image building service or BinderHub. + +(image-building:learn-more)= +## Learn more + +### How can I get started with my environment configuration? + +See [Get started with repo2docker](https://repo2docker.readthedocs.io/en/latest/start/) for a simple tutorial explaining how to set up your environment repository from scratch. + +### Do my commands run as `root` when I build the image? + +No! Commands that you can control run as a user called `jovyan` and do not have root priveleges. +See [the repo2docker docs](https://repo2docker.readthedocs.io/en/latest/use/actions-and-scripts/) for more information. + +### What configuration files can I use to configure my environment? + +The [repo2docker configuration file guide](https://repo2docker.readthedocs.io/en/latest/configuration/) has a complete list of all configuration files you can use. + +### How can I share my hub environment with others on my hub? + +Environment images are cached automatically, so if you give another hub user the URL of the repository you used to build your environment, when they follow the same process above, it will be launched from the cache rather than re-built. + +### How do I update my hub environment? + +Just update any of the files in your environment repository, and your environment will be re-built the next time you launch a session from it. + +### Can I use another community's pre-existing environment? + +Yes! That's often the simplest choice. See [this repo2docker guide on choosing an environment](https://repo2docker.readthedocs.io/en/latest/use/pathways/) for guidance. + +### Where can I learn more about environment configuration and building? + +The [repo2docker user guide is the best place to learn more in general](https://repo2docker.readthedocs.io/en/latest/use/). + +### Where can I find example repositories? + +The [`binder-examples/` GitHub organization](https://github.com/binder-examples/) has a number of repositories that could be built via repo2docker. diff --git a/user/environment/index.md b/user/environment/index.md new file mode 100644 index 0000000..134e5d0 --- /dev/null +++ b/user/environment/index.md @@ -0,0 +1,21 @@ +# Software environment and resources + +JupyterHub can be configured to allow users to select their own environments before launching an interactive session. +There are a few ways that hub administrators can expose custom environments to their users. Below are a few sections that describe each one. + +```{toctree} +:maxdepth: 1 +choose.md +dynamic-imagebuilding.md +``` + +:::{admonition} Ask a hub administrator if these are not enabled +These guides will only work for community hubs that have this functionality enabled. +::: + +## Why environment choice is useful + +- Choosing from custom environments gives users more flexibility to in their use of tools without requiring a new JupyterHub +- Communities can define their own domain-specific environments. +- Communities can re-use and collaborate with other environment maintainers to avoid the burden of maintaining your own environment. +- Administrators can reduce cloud costs by asking users to request environments with more resources only when they really need it. diff --git a/user/howto/launch-dask-gateway-cluster.md b/user/howto/launch-dask-gateway-cluster.md index fd78c22..688ce9a 100644 --- a/user/howto/launch-dask-gateway-cluster.md +++ b/user/howto/launch-dask-gateway-cluster.md @@ -226,4 +226,4 @@ cluster.close() - *How do I install software packages to my Dask hub environment?* - During code development, use the [Dask distributed built-in scheduler plugin](https://distributed.dask.org/en/latest/plugins.html?highlight=pipinstall#built-in-scheduler-plugins) `PipInstall` to pip install a set of packages on all Dask workers (see the [Pangeo docs](https://pangeo.io/cloud.html#dask-software-environment) for an example). However, this increases the time to start up each worker. During production, for best performance we advise adding the required software packages to the [Pangeo Docker image](https://pangeo-docker-images.readthedocs.io/en/latest/), ensuring that the key package `dask-gateway` is included (and `dask-labextension` if you require the Dask dashboard) – see [](/admin/howto/environment/update-community-image.md) for guidance. Deploy your custom image to the hub by either contacting your hub admin to [](/admin/howto/configurator.md) or use [](specify-unlisted-image.md#the-bring-your-own-image-option) in the image selection dropdown if available. + During code development, use the [Dask distributed built-in scheduler plugin](https://distributed.dask.org/en/latest/plugins.html?highlight=pipinstall#built-in-scheduler-plugins) `PipInstall` to pip install a set of packages on all Dask workers (see the [Pangeo docs](https://pangeo.io/cloud.html#dask-software-environment) for an example). However, this increases the time to start up each worker. During production, for best performance we advise adding the required software packages to the [Pangeo Docker image](https://pangeo-docker-images.readthedocs.io/en/latest/), ensuring that the key package `dask-gateway` is included (and `dask-labextension` if you require the Dask dashboard) – see [](/admin/howto/environment/update-community-image.md) for guidance. Deploy your custom image to the hub by either contacting your hub admin to [](/admin/howto/configurator.md) or use [](#environment:bring-your-own-image) in the image selection dropdown if available. diff --git a/user/topics/dynamic-imagebuilding/index.md b/user/topics/dynamic-imagebuilding/index.md deleted file mode 100644 index cec5841..0000000 --- a/user/topics/dynamic-imagebuilding/index.md +++ /dev/null @@ -1,51 +0,0 @@ -# Dynamic image building - -Dynamic image building integrates the capabilities of [BinderHub](https://binderhub.readthedocs.io/en/latest/) with a JupyterHub. This flexibility enables users to manage their own software image environments in the cloud. See the [blog post](https://2i2c.org/blog/2024/jupyterhub-binderhub-gesis/) for more details. - -## Quickstart - -This is a quick guide to getting started with dynamic image building. Note that [repo2docker](https://repo2docker.readthedocs.io/en/latest/) is the underlying technology that powers dynamic image building. This feature presently works with GitHub repositories only. - - - -### Step 1: Prepare a GitHub Repository - -1. Create a **public GitHub repository**. -2. Add the following optional files (choose what you need): - - * `environment.yml` – for `conda` packages (and/or `pip` Python packages). - * `requirements.txt` – for `pip` Python packages. - * `install.R` – for R packages. - * `apt.txt` – for Ubuntu system packages. - * `runtime.txt` – to pin Python/R versions if not specified elsewhere. - * `postBuild` (executable) – script to run after installation. - * `start` (executable) – script to run each time the container starts. - - If you include a `Dockerfile`, then `repo2docker` uses this definition and ignores the above files and a regular Docker build is performed. See the [Binder docs](https://mybinder.readthedocs.io/en/latest/tutorials/dockerfile.html) for guidance on advanced configurations. - -:::{note} -Generally it is best practice not to use `environment.yml` and `requirements.txt` together, as this can lead to conflicts. If you need both `conda` and `pip` packages, use `environment.yml` with `pip` packages listed under the `pip:` section. If you need just `pip` packages, use `requirements.txt` only. -::: - -### Step 2: Configure the JupyterHub “Build your own image” Option - -1. In JupyterHub, choose **“Build your own image”** from the **Image** dropdown in the launcher. -2. Enter your GitHub repo name (e.g. [binder-examples/requirements](https://github.com/binder-examples/requirements)) -3. Optionally include a branch, tag or commit to use. By default, the `HEAD` will use the default branch. -4. Click the {bdg-primary}`Build` button to start the build process. You can view the build logs to monitor progress. -5. Once the build is complete, choose the **Resource Allocation** (CPU, RAM) for the server launch. -6. Click {bdg-primary}`Start` to launch your server. - -### Step 3: Use and Share Your Environment - -1. If you re-launch the same repo and **reuse the same image later**, the build is cached by `repo2docker` unless you update your repo. -1. You can also **share your repo link** with others so they can build the same image in another dynamic image building service or BinderHub. - -## Benefits - -* **No Dockerfile expertise needed** — `repo2docker` generates a Dockerfile automatically from commonly used configuration files of popular package managers, but allows advanced builds for those who are comfortable with Dockerfiles. -* **Reproducible** – your environment configuration is fully described in version-controlled files. -* **Portable to Binder** – the same setup works on [mybinder.org](https://mybinder.org) and any BinderHub, as well as other JupyterHubs with dynamic image building.