Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Docker build #38

Merged
merged 8 commits into from Jul 29, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions .dockerignore
@@ -0,0 +1,7 @@
.git
.tox
.vscode
build
*cache*
*/data
*env*
22 changes: 22 additions & 0 deletions Dockerfile
@@ -0,0 +1,22 @@
FROM python:3.6-slim-stretch
RUN apt-get update \
&& apt-get install -y \
apt-utils \
build-essential \
git \
wget \
curl \
unzip

WORKDIR /app
COPY . /app
ARG TUTORIAL
RUN wget -O snorkel-requirements.txt \
https://raw.githubusercontent.com/HazyResearch/snorkel/redux/requirements.txt \
&& pip3 install -r $TUTORIAL/requirements.txt \
&& pip3 install -r requirements.txt \
&& pip3 install -r snorkel-requirements.txt \
&& python3 -m spacy download en_core_web_sm

WORKDIR $TUTORIAL
ENTRYPOINT ["jupyter", "notebook", "--ip=0.0.0.0", "--no-browser", "--allow-root"]
40 changes: 35 additions & 5 deletions README.md
Expand Up @@ -15,25 +15,55 @@ We recommend that all users **start with the `spam` tutorial** for a gentle intr
All other tutorials assume that you have already completed that tutorial and are familiar with its concepts.

## Getting Started
Running a tutorial has four required steps:

1. Cloning this repo
Step one is cloning this repo.
If you're looking to quickly get started with a tutorial, we recommend using our [Docker setup](#docker).
If you want to install things yourself, you can follow our [installation steps](#install) below instead.

### <a name="docker"> Running with Docker </a>

We've included a Docker setup for our tutorials to make setup easy.
First, make sure you have [Docker installed](https://docs.docker.com/install/) on your machine.
To build and run a Docker image for a tutorial, use `scripts/docker_launch.py` with the `--build` flag.
For example, run the following for the `spam` tutorial:

```bash
python3 scripts/docker_launch.py spam --build
```

This will make a Jupyter notebook server available on port 8888
(you can change the port with the `--port` command line option)
and print out a link you can follow to access the browser interface.
In your browser, open a `.ipynb` file you would like to run &mdash;
such as `spam_tutorial.ipynb` &mdash; and execute the cells in sequence.

Once an image has been built, you can run it without the `--build` flag:

```bash
python3 scripts/docker_launch.py spam
```

### <a name="install"> Installing yourself </a>

Running a tutorial has three required steps if you're installing yourself:

1. Installing repo-wide requirements
1. Installing tutorial-specific requirements
1. Launching a Jupyter notebook server or executing as a script

We recommend installing requirements in a virtual environment using [`virtualenv`](https://virtualenv.pypa.io/en/latest/) or [`conda`](https://docs.conda.io/en/latest/).
For example, if you use pip, first activate your virtualenv if you're using one, then run:
```

```bash
# Install requirements (both shared and tutorial-specific)
pip3 install -r requirements.txt
pip3 install -r spam/requirements.txt

# Launch the Jupyter notebook interface
jupyter notebook
```
Then, in the browser tab that opens, navigate to a `.ipynb` file you would like to run, such as `snorkel-tutorials/spam/spam_tutorial.ipynb`, and click to open it.
Then execute the cells in sequence.
Then in the browser tab that opens, navigate to a `.ipynb` file you would like to run &mdash;
such as `spam/spam_tutorial.ipynb` &mdash; and execute the cells in sequence.

Alternatively, you can run the tutorial as a script by calling `python3` on the corresponding `.py` file directly (e.g. `python3 spam_tutorial.py`).

Expand Down
62 changes: 62 additions & 0 deletions scripts/docker_launch.py
@@ -0,0 +1,62 @@
import argparse
import errno
import os
import socket
import subprocess


def check_docker() -> None:
try:
subprocess.run(["docker", "--version"], check=True)
except (FileNotFoundError, subprocess.CalledProcessError):
raise ValueError("Error calling Docker. Is it installed?")


def build_image(tutorial_name: str) -> None:
arg = f"TUTORIAL={tutorial_name}"
tag = f"--tag={tutorial_name}"
subprocess.run(["docker", "build", "--build-arg", arg, tag, "."], check=True)


def check_port(port: int) -> None:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.bind(("127.0.0.1", port))
except socket.error as e:
if e.errno == errno.EADDRINUSE:
raise ValueError(f"Port {port} is already in use")
else:
raise e


def run_image(tutorial_name: str, port: int) -> None:
tag = f"{tutorial_name}:latest"
p_cfg = f"{port}:{port}"
p_arg = f"--port={port}"
check_run = subprocess.run(["docker", "run", "-it", "-p", p_cfg, tag, p_arg])
if check_run.returncode:
raise ValueError(
"Error running container. If you haven't built it yet, "
"try running this script with the --build flag."
)


def docker_launch(tutorial_name: str, build: bool, port: int) -> None:
if not os.path.isdir(tutorial_name):
raise ValueError(f"{tutorial_name} is not a valid tutorial")
check_docker()
if build:
build_image(tutorial_name)
check_port(port)
run_image(tutorial_name, port)


if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Process some integers.")
parser.add_argument("tutorial_name", help="Name of the tutorial (directory)")
parser.add_argument(
"--build", action="store_true", default=False, help="Build the Docker image?"
)
parser.add_argument("--port", type=int, default=8888, help="Jupyter port for host")
args = parser.parse_args()
docker_launch(**vars(args))