Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add miniforge3 simple base image #129

Merged
merged 12 commits into from
Mar 26, 2020
Merged

Add miniforge3 simple base image #129

merged 12 commits into from
Mar 26, 2020

Conversation

yuvipanda
Copy link
Contributor

@yuvipanda yuvipanda commented Mar 6, 2020

A slim miniforge3 image, equivalent to miniconda3 but with some
slightly different choices.

  • Use ubuntu LTS as base instead of debian
  • Keep $HOME clean, so folks can mount volumes into it

Ref conda-forge/miniforge#20

TODO:

  • Fully understand shell implications
  • Set up automated build
  • Set up some simple tests
  • Set up automated push on merge and tag

A slim miniforge3 image, equivalent to miniconda3 but with some
slightly different choices.

- Use ubuntu LTS as base instead of debian
- Keep $HOME clean, so folks can mount volumes into it

Ref conda-forge/miniforge#20
@yuvipanda
Copy link
Contributor Author

This is currently 286M, compared to 430M for miniconda3

We just need the docker commandline tool, but using
python3 since we might want py.test later
- Put activate commands in user's .bashrc, since this is where
  PS1 is set. If we put the activate commands in /etc/bash.bashrc,
  the environments are set up properly but PS1 is not. This
  is very confusing.
- Put activate commands in /etc/skel as well, so any new non-root
  users created also get base activated by default
- Stop using 'login' shells, as they are no longer needed.
@yuvipanda
Copy link
Contributor Author

base is now visibly activated for root and new non-root users created.

@yuvipanda
Copy link
Contributor Author

I'm going to write tests for:

  1. Validating that $HOME is as empty as it should be
  2. non-root user creation + running works properly

@yuvipanda yuvipanda changed the title Add miniforge3 simple base image [WIP] Add miniforge3 simple base image Mar 6, 2020
@yuvipanda yuvipanda changed the title [WIP] Add miniforge3 simple base image Add miniforge3 simple base image Mar 8, 2020
@yuvipanda
Copy link
Contributor Author

Ok, so the tests I wanted are in now.

How does tagging work for miniforge releases? Is that done manually or automatically? Ideally for miniforge, I'd like automatic releases on miniforge tagging. Maybe that means this should be in the miniforge repo and not here? Happy to move it if that's what is needed.

Excited for this to get going!

@yuvipanda
Copy link
Contributor Author

I do wanna add a test that checks if (base) is activated even when $HOME is an empty volume with no .bashrc, since that is what happens when you use it in a JupyterHub style setting.

@@ -0,0 +1,44 @@
FROM ubuntu:bionic
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does it matter that it is ubuntu over debian?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The primary reason is that Ubuntu has timed long term support releases (every 2y), while debian does not. When people want to use this base image, but add new packages from apt, it should ideally be as up to date as possible. Longer term, this is more likely with Ubuntu than Debian, primarily because of the release cadence. You can get around this to some extent with using their rolling releases, but that makes reproducibility very hard by default.

Jupyter's docker-stacks base images switched from debian to ubuntu in jupyter/docker-stacks#428 for these reasons. repo2docker (and hence binder) use ubuntu as base default for these reasons as well.

Some of this is definitely personal preference of course, so am happy to move to debian if people would like to - although I'd personally prefer to not!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAC (Mandatory Access Control) > Implementations:

  • Debian: seccomp, optional SELinux w/ incomplete policy set
  • Ubuntu: AppArmor, seccomp, optional SELinux w/ incomplete policy set
  • Fedora/CentOS/RHEL: seccomp, SELinux in enforcing mode
  • Android (aarch64/x86; termux): seccomp, SELinux in enforcing mode

yum:

FROM centos:7
# RUN yum update -y
RUN yum install -y wget bzip2 ca-certificates git && yum clean all && rm -rf /var/cache/yum

dnf:

FROM centos:8
# FROM fedora:32
# ...
# RUN dnf update -y
RUN dnf install -y wget bzip2 ca-certificates git && dnf clean all && rm -rf /var/cache/yum /var/cache/dnf

for f in /tmp/tests/*; do \
echo "Executing $f"; \
$f; \
done
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the files here seem to be lacking a single newline at the end

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

@scopatz
Copy link
Member

scopatz commented Mar 9, 2020

Thanks for putting this in @yuvipanda! CC @conda-forge/core

@yuvipanda
Copy link
Contributor Author

Glad to help, @scopatz!

@scopatz
Copy link
Member

scopatz commented Mar 13, 2020

Thanks @yuvipanda! CC @isuruf and @jakirkham for thoughts

@xhochy
Copy link
Member

xhochy commented Mar 26, 2020

I like this and would also use it. For efficiency reasons, could we merge all RUN commands into a single layer?

@jaimergp
Copy link
Member

This post contains some tips for reduced conda docker images too: https://jcristharif.com/conda-docker-tips.html

@jakirkham
Copy link
Member

@xhochy, we already squash all images as part of the build process on CI.

@xhochy
Copy link
Member

xhochy commented Mar 26, 2020

@xhochy, we already squash all images as part of the build process on CI.

Awesome! @jakirkham I would then merge, any objections?

@scopatz
Copy link
Member

scopatz commented Mar 26, 2020

We can always improve it later. Let's go ahead and merge 😉

@scopatz scopatz merged commit f73bdd7 into conda-forge:master Mar 26, 2020
@CJ-Wright
Copy link
Member

This wasn't uploaded to the best of my knowledge. It seems it needs an entry in the .travis.yaml to be uploaded

@xhochy
Copy link
Member

xhochy commented Apr 9, 2020

This wasn't uploaded to the best of my knowledge. It seems it needs an entry in the .travis.yaml to be uploaded

Made a PR at #131

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants