Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# CODEOWNERS file (from GitHub template at
# https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners)
# Each line is a file pattern followed by one or more owners.

################################################################################
# These owners will be the default owners for everything in the repo. This is commented
# out in favor of using a team as the default (see below). It is left here as a comment
# to indicate the primary expert for this code.
# * @ghukill

# Teams can be specified as code owners as well. Teams should be identified in
# the format @org/team-name. Teams must have explicit write access to the
# repository.
* @mitlibraries/dataeng

# Infra should always been involved with changes to the build workflows
/.github/dev-build.yml @mitlibraries/infraeng
/.github/stage-build.yml @mitlibraries/infraeng
/.github/prod-promote.yml @mitlibraries/infraeng

# We set the senior engineer in the team as the owner of the CODEOWNERS file as
# a layer of protection for unauthorized changes.
/.github/CODEOWNERS @ghukill
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -155,3 +155,5 @@ cython_debug/
.DS_Store
output/
.vscode/

AGENTS.md
37 changes: 35 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -70,15 +70,48 @@ black-apply: # Apply changes with 'black'
ruff-apply: # Resolve 'fixable errors' with 'ruff'
uv run ruff check --fix .


####################################
# CLI
####################################
cli-test-inline-run:
uv run python -m launcher.cli \
run \
--mount=tests/fixtures/inline_deps

cli-test-reqs-txt-run:
uv run python -m launcher.cli \
run \
--mount=tests/fixtures/static_deps_reqs_txt \
--requirements=requirements.txt

cli-test-token-authenticated:
uv run python -m launcher.cli \
run \
--mount=tests/fixtures/inline_deps \
--token="iamsecret"

####################################
# Docker
####################################
build: # Build local image for testing
docker-build: # Build local image for testing
docker build -t marimo-launcher:latest .

shell: # Shell into local container for testing
docker-shell: # Shell into local container for testing
docker run -it --entrypoint='bash' marimo-launcher:latest

docker-test-run: # Test local docker container with test fixture notebook
docker run \
-p "2718:2718" \
-v "$(CURDIR)/tests/fixtures:/tmp/fixtures" \
-e NOTEBOOK_MOUNT="/tmp/fixtures" \
-e NOTEBOOK_PATH="helloworld.py" \
marimo-launcher:latest \
run

####################################
# Terraform
####################################

### Terraform-generated Developer Deploy Commands for Dev environment ###
dist-dev: ## Build docker container (intended for developer-based manual build)
Expand Down
137 changes: 136 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,82 @@
- To lint the repo: `make lint`
- To run the app: `uv run launcher --help`

## Overview

This CLI is an application to launch _other_ Marimo notebooks. There are two ways in which this CLI can launch a notebook:

1. The notebook is available on the same machine as the CLI, e.g. mounted into the Docker container
2. A Github repository is passed and cloned by the CLI that contains a notebook

Because this CLI is meant to launch notebooks, it does not have a dedicated ECS task or service.

Take a fictional example of a notebook called "Analyze All the Things (AATT)" in the repository `marimo-aatt`. To provide this notebook for use, an ECS task would be created that sets two important environment variables:

- `NOTEBOOK_REPOSITORY=https://github.com/MITLibraries/marimo-aatt`
- `NOTEBOOK_PATH=aatt.py` (a non-default notebook path)

The ECS task / service would invoke this `marimo-launcher` CLI, and this CLI would perform the following:

1. Clone the Github repository into the container
2. Install dependencies
3. Launch the notebook `aatt.py`

More information about structuring notebooks and dependencies below in "Preparing Notebooks".

## Preparing Notebooks

### Notebook Location
This CLI expects two primary things to discover the notebook to launch:

1. The root directory of the notebook project (either mounted or a cloned Github repository)
2. Path to the actual notebook python file to run

The root of the notebook directory is set either by CLI arg `--repo` / env var `NOTEBOOK_REPOSITORY` or CLI arg `--mount` / env var `NOTEBOOK_MOUNT` (less common, more for dev work). In either approach, a notebook directory is established and all other filepaths -- e.g. notebook or requirements -- are **relative** to this path.

The default notebook path is `notebook.py` and is expected in the root of the cloned or mounted notebook repository. The CLI arg `--path` or env var `NOTEBOOK_PATH` can be passed to override this.

### Notebook Dependencies

There are two primary ways to handle dependencies for a notebook launched by this CLI:

1. Inline dependencies
2. External dependencies requirement file

#### 1- Inline dependencies

This is the **default** behavior for this CLI.

Python [PEP 723](https://peps.python.org/pep-0723/) introduced inline dependencies for a python file. Marimo [fully supports this](https://docs.marimo.io/guides/package_management/inlining_dependencies/) for notebooks as well.

Inline dependencies are a text block at the top of the python notebook that outline what dependencies should be installed. This section looks and feels much like sections in the `pyproject.toml`. Here is a minimal example from `tests/fixtures/inline_deps/notebook.py`:

```python
# /// script
# requires-python = ">=3.13"
# dependencies = [
# "marimo",
# "tinydb==4.8.2",
# ]
# ///

# rest of notebook here...
```

When the CLI launches this notebook it will include the flag `--sandbox` when running Marimo that instructs Marimo to use the inlined dependencies.

The `Makefile` command `cli-test-inline-run` will demonstrate this.

#### 2- External dependencies requirement file

Another option, which requires the CLI flag `--requirements` or env var `NOTEBOOK_REQUIREMENTS`, is to install dependencies found in a standalone requirements file, e.g. `requirements.txt`. The tests fixture `tests/fixtures/static_deps_reqs_txt/requirements.txt` shows an example of this kind of file.

The flag `--requirements` or env var `NOTEBOOK_REQUIREMENTS` should point to a relative path from the root of the notebook directory where this file can be found. When passed, Marimo will be launched with the flag `--with-requirements` which instructs it to created an isolated environment with these dependencies.

There are many ways to create this file, [`uv export` is worth consideration](https://docs.astral.sh/uv/reference/cli/#uv-export).

The `Makefile` command `cli-test-reqs-txt-run` will demonstrate this.


## Environment Variables

### Required
Expand All @@ -20,7 +96,66 @@ WORKSPACE=### Set to `dev` for local development, this will be set to `stage` an

### Optional

None yet...
Set these if you want to override defaults or pass values via env instead of flags. Keep them unset if you use CLI options.

```shell
NOTEBOOK_REPOSITORY= ### repository to clone that contains a notebook and any required assets
NOTEBOOK_REPOSITORY_BRANCH= ### optional branch to checkout on clone
NOTEBOOK_MOUNT= ### either local of Docker context, an accessible root directory that contains notebook(s)
NOTEBOOK_PATH=### Relative path of actual notebook .py file based on cloned repository or mounted directory; defaults to "notebook.py"
NOTEBOOK_REQUIREMENTS= ### filepath to install dependencies from, relative to notebook root; if unset assuming dependencies are inline in notebook

NOTEBOOK_MODE= ### how to launch marimo: "run" to execute, "edit" to open the editor; default "run"
NOTEBOOK_HOST= ### host to bind running notebook to
NOTEBOOK_PORT= ### port to serve running notebook on
```


## CLI Commands

### `launcher`

Base command

```text
Usage: launcher [OPTIONS] COMMAND [ARGS]...

Options:
-v, --verbose Pass to log at debug level instead of info
--help Show this message and exit.

Commands:
run
validate
```

### `launcher run`

```text
Usage: python -m launcher.cli run [OPTIONS]

Launch notebook in 'run' or 'edit' mode.

Options:
--mount PATH path to mounted / existing notebook directory (env:
NOTEBOOK_MOUNT)
--repo TEXT git repository URL containing the notebook (env:
NOTEBOOK_REPOSITORY)
--repo-branch TEXT optional branch to checkout from cloned notebook
repository (env: NOTEBOOK_REPOSITORY_BRANCH)
--path TEXT relative path to the notebook within the directory
(env: NOTEBOOK_PATH)
--requirements PATH path to requirements file for environment (env:
NOTEBOOK_REQUIREMENTS)
--mode [run|edit] launch mode, 'run' or 'edit' (env: NOTEBOOK_MODE)
[default: run]
--host TEXT host interface to bind (env: NOTEBOOK_HOST) [default:
0.0.0.0]
--port INTEGER port to bind (env: NOTEBOOK_PORT) [default: 2718]
--token TEXT set a required authentication token/password for the
notebook; if not set, no token/password is required
--help Show this message and exit.
```



Expand Down
Loading