Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] Preliminary Ubuntu 20.04 images for both CPU and GPU as well as docs #312

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions docker/GenCPU.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
FROM ubuntu:20.04

LABEL maintainer="MIT Probabilistic Computing Project"

# Find current Julia version on https://julialang.org/downloads/
ARG JULIA_VERSION_SHORT="1.5"
ARG JULIA_VERSION_FULL="${JULIA_VERSION_SHORT}.1"
ENV JULIA_INSTALLATION_PATH=/opt/julia
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate - see line 11.


ENV DEBIAN_FRONTEND=noninteractive
ENV JULIA_INSTALLATION_PATH=/opt/julia

RUN apt-get update -qq \
&& apt-get install -qq -y --no-install-recommends\
build-essential \
ca-certificates \
curl \
ffmpeg \
git \
graphviz \
hdf5-tools \
python3-dev \
python3-pip \
python3-tk \
rsync \
software-properties-common \
wget \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/* && \
ln -s /usr/bin/python3 /usr/bin/python

RUN wget https://julialang-s3.julialang.org/bin/linux/x64/${JULIA_VERSION_SHORT}/julia-${JULIA_VERSION_FULL}-linux-x86_64.tar.gz && \
tar zxf julia-${JULIA_VERSION_FULL}-linux-x86_64.tar.gz && \
mkdir -p "${JULIA_INSTALLATION_PATH}" && \
mv julia-${JULIA_VERSION_FULL} "${JULIA_INSTALLATION_PATH}/" && \
ln -fs "${JULIA_INSTALLATION_PATH}/julia-${JULIA_VERSION_FULL}/bin/julia" /usr/local/bin/ && \
rm julia-${JULIA_VERSION_FULL}-linux-x86_64.tar.gz && \
julia -e 'import Pkg; Pkg.add("IJulia")'

RUN julia -e 'import Pkg; Pkg.add(["Gen"])'
40 changes: 40 additions & 0 deletions docker/GenGPU.dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
FROM nvidia/opengl:1.0-glvnd-runtime-ubuntu20.04

LABEL maintainer="MIT Probabilistic Computing Project"

# Find current Julia version on https://julialang.org/downloads/
ARG JULIA_VERSION_SHORT="1.5"
ARG JULIA_VERSION_FULL="${JULIA_VERSION_SHORT}.1"
ENV JULIA_INSTALLATION_PATH=/opt/julia
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate - see line 11.


ENV DEBIAN_FRONTEND=noninteractive
ENV JULIA_INSTALLATION_PATH=/opt/julia

RUN apt-get update -qq \
&& apt-get install -qq -y --no-install-recommends\
build-essential \
ca-certificates \
curl \
ffmpeg \
git \
graphviz \
hdf5-tools \
python3-dev \
python3-pip \
python3-tk \
rsync \
software-properties-common \
wget \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/* && \
ln -s /usr/bin/python3 /usr/bin/python

RUN wget https://julialang-s3.julialang.org/bin/linux/x64/${JULIA_VERSION_SHORT}/julia-${JULIA_VERSION_FULL}-linux-x86_64.tar.gz && \
tar zxf julia-${JULIA_VERSION_FULL}-linux-x86_64.tar.gz && \
mkdir -p "${JULIA_INSTALLATION_PATH}" && \
mv julia-${JULIA_VERSION_FULL} "${JULIA_INSTALLATION_PATH}/" && \
ln -fs "${JULIA_INSTALLATION_PATH}/julia-${JULIA_VERSION_FULL}/bin/julia" /usr/local/bin/ && \
rm julia-${JULIA_VERSION_FULL}-linux-x86_64.tar.gz && \
julia -e 'import Pkg; Pkg.add("IJulia")'

RUN julia -e 'import Pkg; Pkg.add(["Gen"])'
41 changes: 41 additions & 0 deletions docker/Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Gen Containerization
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps rather than in its own markdown file, this should be added as a subsection at the bottom of docs/src/getting_started.md. I say bottom because I think Gen wants to present primarily as an ordinary Julia package rather than a monolithic stack (perhaps @marcoct can comment). Also, we should include in the beginning of this section an explanation of why the user might ever want to use these container instructions (e.g., for portability / saves the need to install anything other than Docker).


## Quickstart

### Docker Setup
Please follow the [official Docker installation instructions](https://docs.docker.com/engine/install/) to setup Docker on your target platform. If you would like to use GPU acceleration, please also install the [NVIDIA Container Toolkit / nvidia-docker](https://github.com/NVIDIA/nvidia-docker).


### Building Container
You can build Docker containers for Gen that either target CPU-only or GPU-enabled machines.
- Please run `docker build -f docker/GenCPU.dockerfile -t gen:cpu-ubuntu20.04 .` for the CPU image
- Please run `docker build -f docker/GenGPU.dockerfile -t gen:gpu-ubuntu20.04 .` if you want to use GPU acceleration
Both commands are supposed to be run from the main Gen.jl folder, not this subfolder.
Comment on lines +9 to +13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should have the "easy version" be to pull a pre-built image from Docker Hub, and have building the container on one's own be the "advanced version"?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely, I just lack access rights to do that for you. But I would highly recommend that as well. docker run -it probcomp/gen:cpu-ubuntu20.04 bash should just work, absolutely.


### Running Container
You can run `docker run -it gen:cpu-ubuntu20.04 bash` to run the CPU image and `docker run -it --gpus all gen:gpu-ubuntu20.04 bash` to run the GPU image. Afterwards, you can just execute `julia`, type `using Gen` and start developing with Gen. If you would like to remove the container, please add the flag `--rm` to your run command.

## Known Issues

### Run or Build Script Not Executable
If you get an error like
```
bash: ./docker_build_gpu.sh: Permission denied
```
Then please make the script executable via `chmod +x docker_build_gpu.sh` (analogous for CPU).

### Known Issues with Docker on macOS
The Docker container might currently fail on macOS with the following error:
```
julia> using Gen
[ Info: Precompiling Gen [ea4f424c-a589-11e8-07c0-fd5c91b9da4a]
ERROR: Failed to precompile Gen [ea4f424c-a589-11e8-07c0-fd5c91b9da4a] to /root/.julia/compiled/v1.4/Gen/OEZG1_t5nDi.ji.
Stacktrace:
[1] error(::String) at ./error.jl:33
[2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1272
[3] _require(::Base.PkgId) at ./loading.jl:1029
[4] require(::Base.PkgId) at ./loading.jl:927
[5] require(::Module, ::Symbol) at ./loading.jl:922
```

This can even happen when running inside a VM on macOS. While this bug is confirmed and reproducable, there is currently no solution for it, cmp. [issue 311](https://github.com/probcomp/Gen.jl/issues/311).
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo - should be "reproducible"

33 changes: 0 additions & 33 deletions docker/ubuntu1604

This file was deleted.