Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] Preliminary Ubuntu 20.04 images for both CPU and GPU as well as docs #312

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

fplk
Copy link

@fplk fplk commented Sep 18, 2020

  • Current image in master will not build at all due to wget https://julialang.org/juliareleases.asc. That will not resolve, since this file was moved to https://julialang.org/assets/juliareleases.asc The PRed images work.
  • Currently, there are no container instructions - neither in the Dockerfile nor in the Github readme nor in the Gen docs, so I have added docs to build and run the image
  • Images for CPU and GPU
  • Updates Julia 0.7 (from 2018) to 1.5.1 (current)
  • Updates Ubuntu 16.04 to 20.04, since 16.04 is in ESM - cmp. release cycle whereas 20.04 received full hardware & maintenance updates until 2022, LTS until 4/2025 and ESM until 04/2030
  • Tweaks: Updates MAINTAINER tag which is deprecated to LABEL, removes cached files and uses --no-install-recommends for smaller image size, reduces number of layers caused by separate RUN lines in current image
  • Forms foundation to derive other Gen project images from (I will soon push POCs for 1 external and 2 internal projects)

We might consider:

  • Adding JupyterLab (which I already have in internal image)
  • Consolidating this with the gen-quickstart image (which I have also contributed)
  • First make sure that internal projects can successfully base their containers on these before merging
  • Further investigate issue 311 before merging, since this currently blocks using Gen in containers on macOS
  • Could add checksum verification back in like the previous image did

Happy to accommodate change requests - looking forward to feedback.

# Find current Julia version on https://julialang.org/downloads/
ARG JULIA_VERSION_SHORT="1.5"
ARG JULIA_VERSION_FULL="${JULIA_VERSION_SHORT}.1"
ENV JULIA_INSTALLATION_PATH=/opt/julia
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate - see line 11.

# Find current Julia version on https://julialang.org/downloads/
ARG JULIA_VERSION_SHORT="1.5"
ARG JULIA_VERSION_FULL="${JULIA_VERSION_SHORT}.1"
ENV JULIA_INSTALLATION_PATH=/opt/julia
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate - see line 11.

[5] require(::Module, ::Symbol) at ./loading.jl:922
```

This can even happen when running inside a VM on macOS. While this bug is confirmed and reproducable, there is currently no solution for it, cmp. [issue 311](https://github.com/probcomp/Gen.jl/issues/311).
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo - should be "reproducible"

Copy link
Contributor

@bzinberg bzinberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me like duplicating the Dockerfiles to support both CPU and GPU is currently overkill, since none of Gen's native operations use GPU and (afaik) no one has ever asked us for help with GPU-related operations. I guess it's ok for now, since it's not that much more work to have the second image.

Comment on lines +9 to +13
### Building Container
You can build Docker containers for Gen that either target CPU-only or GPU-enabled machines.
- Please run `docker build -f docker/GenCPU.dockerfile -t gen:cpu-ubuntu20.04 .` for the CPU image
- Please run `docker build -f docker/GenGPU.dockerfile -t gen:gpu-ubuntu20.04 .` if you want to use GPU acceleration
Both commands are supposed to be run from the main Gen.jl folder, not this subfolder.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should have the "easy version" be to pull a pre-built image from Docker Hub, and have building the container on one's own be the "advanced version"?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely, I just lack access rights to do that for you. But I would highly recommend that as well. docker run -it probcomp/gen:cpu-ubuntu20.04 bash should just work, absolutely.

@@ -0,0 +1,41 @@
# Gen Containerization
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps rather than in its own markdown file, this should be added as a subsection at the bottom of docs/src/getting_started.md. I say bottom because I think Gen wants to present primarily as an ordinary Julia package rather than a monolithic stack (perhaps @marcoct can comment). Also, we should include in the beginning of this section an explanation of why the user might ever want to use these container instructions (e.g., for portability / saves the need to install anything other than Docker).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants