NVIDIA · samcmill · Dec 10, 2018 · Nov 30, 2018 · Nov 30, 2018 · Dec 3, 2018
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,11 +1,11 @@
 include CHANGELOG.md
 include LICENSE
 include README.md
-include RECIPES.md
 include hpccm.py
 include test/bad_recipe.py
 include test/docker.blob
 include test/global_vars_recipe.py
 include test/helpers.py
 include test/singularity.blob
+graft docs
 graft recipes
diff --git a/README.md b/README.md
@@ -1,299 +1,43 @@
 # HPC Container Maker
 
-## Introduction
-
-HPC Container Maker (HPCCM) generates container specification files,
-either Dockerfiles or Singularity recipe files, based on a "recipe."
-A recipe specifies the series of steps to be performed when building a
-container.  The recipe format is described [elsewhere](RECIPES.md).
-Recipes provide more portable, higher level building blocks that
-separate the concerns of choosing what to include in a container from
-the low level details of container specification.
-
-## Why use container maker?
-
-At first glance, the idea of specifying a container in a new,
-non-native format may not seem worthwhile.  For small, simple
-containers, this is probably true.  However, for larger, more complex
-containers, there are 3 good reasons to use Container Maker recipes.
-
- 1. [Container implementation abstraction](#container-implementation-abstraction)
-
- 2. [Availability of a full programming language](#availability-of-a-full-programming-language)
-
- 3. [Higher level abstraction](#higher-level-abstraction), i.e., building blocks
-
-Recipes also address the combinatorial explosion problem as the number
-of components and versions increase.
-
-And as a bonus, there is absolutely no lock in.  Container Maker
-generates human readable Dockerfiles and Singularity recipe files, so
-you can always decide to revert to writing in a native container
-specification format.
-
-### Container implementation abstraction
-
-The same recipe file generates specification files for Docker or
-Singularity.  There is no need for separate development for each
-container implementation you want to natively support.
-
-For example, from this [single recipe](recipes/examples/basic.py)
-both a Dockerfile and Singularity recipe file can be easily generated.
-(Don't worry about the [syntax of a recipe](RECIPES.md) right now,
-although as you can see it's straight forward.)
-
-```python
-Stage0.baseimage('ubuntu:16.04')
-Stage0 += apt_get(ospackages=['gcc', 'g++', 'gfortran'])
-```
-
-```
-FROM ubuntu:16.04
-
-RUN apt-get update -y && \
-    apt-get install -y --no-install-recommends \
-        gcc \
-        g++ \
-        gfortran && \
-    rm -rf /var/lib/apt/lists/*
-```
-
-```
-BootStrap: docker
-From: ubuntu:16.04
-
-%post
-    apt-get update -y
-    apt-get install -y --no-install-recommends \
-        gcc \
-        g++ \
-        gfortran
-    rm -rf /var/lib/apt/lists/*
-```
-
-### Availability of a full programming language
-
-A recipe is Python code.  This means that you can use the full power
-of Python in a recipe for conditional branching, input validation,
-searching the web for the latest version of a component, etc.
-
-For example, the LAMMPS application may be built in single, double, or
-mixed precision mode.  The following will select and the validate the
-precision based on a user supplied argument (`hpccm.py --userarg
-LAMMPS_PRECISION=...`).
-
-```python
-# get and validate precision
-VALID_PRECISION = ['single', 'double', 'mixed']
-precision = USERARG.get('LAMMPS_PRECISION', 'single')
-if precision not in VALID_PRECISION:
-    raise ValueError('Invalid precision')
-
-...
-
-Stage0 += shell(commands=['make -f Makefile.linux.{}'.format(precision), ...])
-
-...
-```
-
-A standalone [example](recipes/examples/userargs.py) is also provided.
-
-### Higher level abstraction
-
-While it is possible to write a recipe that corresponds nearly one for
-one with a Dockerfile or Singularity recipe file, the Container Maker
-provides higher level abstractions called building blocks to simplify
-recipes and encapsulate best practices.
-
-The first reason illustrates a simple case of installing OS packages.
-OpenMPI is a more significant example; with a single recipe line, a
-series of optimized Docker or Singularity instructions are generated.
-
-```python
-Stage0 += openmpi(cuda=True, infiniband=True,
-                  prefix='/usr/local/openmpi', version='3.0.0')
-```
-
-```
-# OpenMPI version 3.0.0
-RUN apt-get update -y && \
-    apt-get install -y --no-install-recommends \
-        file \
-        hwloc \
-        openssh-client \
-        wget && \
-    rm -rf /var/lib/apt/lists/*
-RUN mkdir -p /tmp && wget -q --no-check-certificate -P /tmp https://www.open-mpi.org/software/ompi/v3.0/downloads/openmpi-3.0.0.tar.bz2 && \
-    tar -x -f /tmp/openmpi-3.0.0.tar.bz2 -C /tmp -j && \
-    cd /tmp/openmpi-3.0.0 &&   ./configure --prefix=/usr/local/openmpi --disable-getpwuid --enable-orterun-prefix-by-default --with-cuda --with-verbs && \
-    make -j4 && \
-    make -j4 install && \
-    rm -rf /tmp/openmpi-3.0.0.tar.bz2 /tmp/openmpi-3.0.0
-ENV PATH=/usr/local/openmpi/bin:$PATH \
-    LD_LIBRARY_PATH=/usr/local/openmpi/lib:$LD_LIBRARY_PATH
-```
-
-```
-# OpenMPI version 3.0.0
-%post
-    apt-get update -y
-    apt-get install -y --no-install-recommends \
-        file \
-        hwloc \
-        openssh-client \
-        wget
-    rm -rf /var/lib/apt/lists/*
-%post
-    mkdir -p /tmp && wget -q --no-check-certificate -P /tmp https://www.open-mpi.org/software/ompi/v3.0/downloads/openmpi-3.0.0.tar.bz2
-    tar -x -f /tmp/openmpi-3.0.0.tar.bz2 -C /tmp -j
-    cd /tmp/openmpi-3.0.0 &&   ./configure --prefix=/usr/local/openmpi --disable-getpwuid --enable-orterun-prefix-by-default --with-cuda --with-verbs
-    make -j4
-    make -j4 install
-    rm -rf /tmp/openmpi-3.0.0.tar.bz2 /tmp/openmpi-3.0.0
-%environment
-    export PATH=/usr/local/openmpi/bin:$PATH
-    export LD_LIBRARY_PATH=/usr/local/openmpi/lib:$LD_LIBRARY_PATH
-```
-
-## Installation
-
-To install the latest stable version:
-
-```
-$ pip install hpccm
-```
-
-To install the current development branch:
-
-```
-$ pip install https://github.com/NVIDIA/hpc-container-maker/tarball/master
-```
-
-In either case, `hpccm` will be installed in your `PATH`.
-
-You can also use `hpccm.py` or `python -m hpccm.cli` from a clone of
-the repository without having to install anything (maybe you have to install the `six`-package: `pip install six`).
-
-The sections below assume `hpccm.py` from a clone of the repository.
-
-## How to Use Container Maker
-
-Typically, Container Maker will be used in 1 of 2 scenarios.
-
-### Template for by-hand customization
-
-In this scenario, a base recipe containing the required core
-components (e.g., OFED, OpenMPI) will be selected or created and used
-to generate a template Dockerfile or Singularity recipe file.  The
-base recipe contains everything needed up to, but not including the
-application specification.  The application specification is then
-added by-hand to the Dockerfile or Singularity recipe file which is
-used to generate the container image.
-
-HPC base recipe files, containing commonly-required core HPC
-components, are included for the [GNU compiler with
-OpenMPI](recipes/hpcbase-gnu-openmpi.py), the [GNU compiler with
-MVAPICH2](recipes/hpcbase-gnu-mvapich2.py), the [PGI compiler with
-OpenMPI](recipes/hpcbase-pgi-openmpi.py), and the [PGI compiler with
-MVAPICH2](recipes/hpcbase-pgi-mvapich2.py).
-
-The workflow for this scenario follows.
-
- 1. Generate a preliminary Dockerfile, e.g., `hpccm.py --recipe recipes/hpcbase-gnu-openmpi.py > Dockerfile`.
-
- 2. Add the HPC application specific build steps to the Dockerfile from
-   step 1.
-
- 3. Build the Docker container, e.g., `docker build -t myapp -f Dockerfile .`.
-
- 4. Run the Docker container, e.g., `nvidia-docker run --rm -ti myapp`.
-
-A variant of this scenario is to use the base recipe to generate a
-base image containing the required core components.  The resulting
-image can then be referenced from a Dockerfile or Singularity recipe
-file.
-
-The workflow for this scenario variant follows.
-
- 1. Generate a base image Dockerfile, e.g., `hpccm.py --recipe recipes/hpcbase-gnu-openmpi.py > Dockerfile.base`.
-
- 2. Generate a Docker image, e.g., `docker build -t base -f Dockerfile.base .`.
-
- 3. Create a Dockerfile that references the base image from step 2 and
-   contains the HPC application specific build steps.
-
- 4. Build the Docker container, e.g., `docker build -t myapp -f Dockerfile .`.
-
- 5. Run the Docker container, e.g., `nvidia-docker run --rm -ti myapp`.
-
-### Full container specification
-
-In this scenario the application itself is included in the Container
-Maker recipe.
-
-Some sample recipe files for applications are included with the tool.
-Note that some of these recipes may depend on certain files in the
-build context and may not build without them.
-
-A sample workflow for this scenario follows.
-
- 1. Generate a Dockerfile, e.g., `hpccm.py --recipe recipes/gromacs/gromacs.py > Dockerfile`
-
- 2. Generate a Docker image, e.g., `docker build -t gromacs -f Dockerfile .`.
-
- 3. Run the Docker container, e.g., `nvidia-docker run --rm -ti gromacs`.
-
-## Creating Singularity Images
-
-There are two ways to create Singularity images, one using Docker and
-the other using Singularity natively.
-
-If container image size is a concern and multi-stage recipes are being
-used, the Docker-based workflow must be used.
-
-### Docker-based workflow
-
- 1. Generate a Dockerfile from the recipe.  E.g., `hpccm.py --recipe recipes/examples/basic.py > Dockerfile`
-
- 2. Build the Docker container. E.g., `docker build -t basic -f Dockerfile .`
-
- 3. Convert the container to a Singularity image.  E.g.,
-`docker run -t --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/output singularityware/docker2singularity basic`
-
-### Singularity native workflow
-
- 1. Generate a Singularity recipe file.  E.g., `hpccm.py --recipe recipes/examples/basic.py --format singularity > Singularity`
-
- 2. Build the Singularity container. E.g., `sudo singularity build basic.simg Singularity`
-
-## Usage
-
-### Command line options
-
-- `--format`: specify the container format to output.  The choices are
-  `docker` and `singularity`; `docker` is the default.
-
-- `--help`: print usage information and exit.
-
-- `--print-exceptions`: rather the print a concise error message,
-  print out the entire stack trace if an error occurs.
-
-- `--single-stage`: process only the first stage of a multi-stage
-  recipe.
-
-- `--recipe`: specify the path to the recipe file.  See the
-  [recipes](recipes) sub-directory for some example recipes.
-
-- `--userarg`: specify one or more key / value pairs that are passed
-  through to the recipe as a dictionary named `USERARG`.
-
-## Current Limitations
-
-- Singularity does not support multi-stage containers.  This is a
-  limitation of Singularity itself, not Container Maker.  Only the
-  first stage of a multi-stage recipe will be generated for
-  Singularity.
-
-## TODO
-
-- Additional building blocks
+HPC Container Maker (HPCCM - pronounced H-P-see-M) is an open source
+tool to make it easier to generate container specification files.
+
+- [Documentation](/docs)
+    - [Getting Started](/docs/getting_started.md)
+    - [Tutorial](/docs/tutorial.md)
+    - [Recipes](/docs/recipes.md)
+    - [Workflows](/docs/workflows.md)
+    - [API: Building Blocks](/docs/building_blocks.md)
+    - [API: Primitives](/docs/primitives.md)
+    - [API: Miscellaneous](/docs/misc_api.md)
+- [Examples](/recipes/)
+- [Citation](/docs/citation.md)
+- [License](/LICENSE)
+
+## Overview
+
+HPC Container Maker generates Dockerfiles or Singularity definition
+files from a high level Python recipe.  HPCCM recipes have some
+distinct advantages over "native" container specification formats.
+
+1. A library of HPC [building blocks](/docs/building_blocks.md) that
+   separate the choice of what to include in a container image from
+   the details of how it's done.  The building blocks transparently
+   provide the latest component and container best practices.
+
+2. Python provides increased flexibility over static container
+   specification formats.  Python-based recipes can branch, validate
+   user input, etc. - the same recipe can generate multiple container
+   specifications.
+
+3. Generate either Dockerfiles or Singularity definition files from
+   the same recipe.
+
+## Additional Resources
+
+- [Making Containers Easier With HPC Container Maker (paper)](https://github.com/HPCSYSPROS/Workshop18/blob/master/Making_Container_Easier_with_HPC_Container_Maker/ws_hpcsysp103.pdf), presented at the [HPC Systems Professionals Workshop at SC18](/docs/citation.md)
+- [Overview presentation at SC18 (video)](http://on-demand.gputechconf.com/supercomputing/2018/video/sc1843-making-containers-easier-hpc-container-maker.html)
+- [Making Containers Easier with HPC Container Maker (webinar)](https://www.nvidia.com/content/webinar-portal/src/webinar-portal.html?D2C=1802760&isSocialSharing=Y&partnerref=emailShareFromGateway)
+- [ADMIN Magazine article](http://www.admin-magazine.com/HPC/Articles/HPC-Container-Maker) by Jeff Layton
+- [NVIDIA Developer Blog](https://devblogs.nvidia.com/making-containers-easier-with-hpc-container-maker/) by Scott McMillan