Skip to content

[ci] Pull base images for CI instead of building them from scratch #1909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 16, 2021
Merged

[ci] Pull base images for CI instead of building them from scratch #1909

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fba9786
Pull base images for CI instead of building from scratch
Apr 2, 2021
3e3dd94
Merge branch 'master' into ci/base_dockerfiles
Apr 7, 2021
53095ed
Address PR comments
Apr 13, 2021
96d3898
Merge branch 'master' into ci/base_dockerfiles
Apr 13, 2021
fc0e076
Merge branch 'ci/base_dockerfiles' of github.com:teojgo/reframe into …
Apr 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions ci-scripts/dockerfiles/Lmod-base.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM ubuntu:20.04

ENV TZ=Europe/Zurich
ENV DEBIAN_FRONTEND=noninteractive
ENV _LMOD_VER=8.4.12

# ReFrame requirements
RUN \
apt-get -y update && \
apt-get -y install ca-certificates && \
update-ca-certificates && \
apt-get -y install gcc make git python3 python3-pip

# Required utilities
RUN apt-get -y install wget

# Install Lmod
RUN \
apt-get -y install lua5.3 lua-bit32:amd64 lua-posix:amd64 lua-posix-dev liblua5.3-0:amd64 liblua5.3-dev:amd64 tcl tcl-dev tcl8.6 tcl8.6-dev:amd64 libtcl8.6:amd64 && \
wget -q https://github.com/TACC/Lmod/archive/${_LMOD_VER}.tar.gz -O lmod.tar.gz && \
tar xzf lmod.tar.gz && \
cd Lmod-${_LMOD_VER} && \
./configure && make install && \
cd .. && rm -rf lmod.tar.gz Lmod-${_LMOD_VER} && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

ENV BASH_ENV=/usr/local/lmod/lmod/init/profile
28 changes: 1 addition & 27 deletions ci-scripts/dockerfiles/Lmod.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,38 +2,12 @@
# Execute this from the top-level ReFrame source directory
#

FROM ubuntu:20.04

ENV TZ=Europe/Zurich
ENV DEBIAN_FRONTEND=noninteractive
ENV _LMOD_VER=8.4.12
FROM reframehpc/rfm-ci-base:lmod

# ReFrame user
RUN useradd -ms /bin/bash rfmuser

# ReFrame requirements
RUN \
apt-get -y update && \
apt-get -y install ca-certificates && \
update-ca-certificates && \
apt-get -y install gcc && \
apt-get -y install make && \
apt-get -y install git && \
apt-get -y install python3 python3-pip

# Required utilities
RUN apt-get -y install wget

# Install Lmod
RUN \
apt-get -y install lua5.3 lua-bit32:amd64 lua-posix:amd64 lua-posix-dev liblua5.3-0:amd64 liblua5.3-dev:amd64 tcl tcl-dev tcl8.6 tcl8.6-dev:amd64 libtcl8.6:amd64 && \
wget -q https://github.com/TACC/Lmod/archive/${_LMOD_VER}.tar.gz -O lmod.tar.gz && \
tar xzf lmod.tar.gz && \
cd Lmod-${_LMOD_VER} && \
./configure && make install

ENV BASH_ENV=/usr/local/lmod/lmod/init/profile

USER rfmuser

# Install ReFrame from the current directory
Expand Down
34 changes: 34 additions & 0 deletions ci-scripts/dockerfiles/Lmod77-base.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
#
# LMod versions prior to 8.2 emitted Python commands differently, so we use this
# Dockerfile to test the bindings of older versions
#


FROM ubuntu:20.04

ENV TZ=Europe/Zurich
ENV DEBIAN_FRONTEND=noninteractive
ENV _LMOD_VER=7.7

# ReFrame requirements
RUN \
apt-get -y update && \
apt-get -y install ca-certificates && \
update-ca-certificates && \
apt-get -y install gcc make git python3 python3-pip

# Required utilities
RUN apt-get -y install wget

# Install Lmod
RUN \
apt-get -y install lua5.3 lua-bit32:amd64 lua-posix:amd64 lua-posix-dev liblua5.3-0:amd64 liblua5.3-dev:amd64 tcl tcl-dev tcl8.6 tcl8.6-dev:amd64 libtcl8.6:amd64 lua-filesystem:amd64 lua-filesystem-dev:amd64 && \
wget -q https://github.com/TACC/Lmod/archive/${_LMOD_VER}.tar.gz -O lmod.tar.gz && \
tar xzf lmod.tar.gz && \
cd Lmod-${_LMOD_VER} && \
./configure && make install && \
cd .. && rm -rf lmod.tar.gz Lmod-${_LMOD_VER} && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

ENV BASH_ENV=/usr/local/lmod/lmod/init/profile
34 changes: 1 addition & 33 deletions ci-scripts/dockerfiles/Lmod77.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,44 +2,12 @@
# Execute this from the top-level ReFrame source directory
#

#
# LMod versions prior to 8.2 emitted Python commands differently, so we use this
# Dockerfile to test the bindings of older versions
#


FROM ubuntu:20.04

ENV TZ=Europe/Zurich
ENV DEBIAN_FRONTEND=noninteractive
ENV _LMOD_VER=7.7
FROM reframehpc/rfm-ci-base:lmod77

# ReFrame user
RUN useradd -ms /bin/bash rfmuser

# ReFrame requirements
RUN \
apt-get -y update && \
apt-get -y install ca-certificates && \
update-ca-certificates && \
apt-get -y install gcc && \
apt-get -y install make && \
apt-get -y install git && \
apt-get -y install python3 python3-pip

# Required utilities
RUN apt-get -y install wget

# Install Lmod
RUN \
apt-get -y install lua5.3 lua-bit32:amd64 lua-posix:amd64 lua-posix-dev liblua5.3-0:amd64 liblua5.3-dev:amd64 tcl tcl-dev tcl8.6 tcl8.6-dev:amd64 libtcl8.6:amd64 lua-filesystem:amd64 lua-filesystem-dev:amd64 && \
wget -q https://github.com/TACC/Lmod/archive/${_LMOD_VER}.tar.gz -O lmod.tar.gz && \
tar xzf lmod.tar.gz && \
cd Lmod-${_LMOD_VER} && \
./configure && make install

ENV BASH_ENV=/usr/local/lmod/lmod/init/profile

USER rfmuser

# Install ReFrame from the current directory
Expand Down
10 changes: 10 additions & 0 deletions ci-scripts/dockerfiles/Tmod32-base.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
FROM centos:7

# ReFrame requirements
RUN \
yum -y install gcc make git python3

# Install Tmod 3.2
RUN yum -y install environment-modules && \
yum clean all && \
rm -rf /var/cache/yum
15 changes: 1 addition & 14 deletions ci-scripts/dockerfiles/Tmod32.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,11 @@
# Execute this from the top-level ReFrame source directory
#

FROM centos:7
FROM reframehpc/rfm-ci-base:tmod32

# ReFrame user
RUN useradd -ms /bin/bash rfmuser

# ReFrame requirements
RUN \
yum -y install gcc && \
yum -y install make && \
yum -y install git && \
yum -y install python3

# # Required utilities
# RUN apt-get -y install wget

# Install Tmod 3.2
RUN yum -y install environment-modules

USER rfmuser

# Install ReFrame from the current directory
Expand Down
28 changes: 28 additions & 0 deletions ci-scripts/dockerfiles/Tmod4-base.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
FROM ubuntu:20.04

ENV TZ=Europe/Zurich
ENV DEBIAN_FRONTEND=noninteractive
ENV _TMOD_VER=4.6.0

# ReFrame requirements
RUN \
apt-get -y update && \
apt-get -y install ca-certificates && \
update-ca-certificates && \
apt-get -y install gcc make git python3 python3-pip

# Required utilities
RUN apt-get -y install wget

# Install Tmod4
RUN \
apt-get -y install autoconf tcl-dev && \
wget -q https://github.com/cea-hpc/modules/archive/v${_TMOD_VER}.tar.gz -O tmod.tar.gz && \
tar xzf tmod.tar.gz && \
cd modules-${_TMOD_VER} && \
./configure && make install && \
cd .. && rm -rf tmod.tar.gz modules-${_TMOD_VER} && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*

ENV BASH_ENV=/usr/local/Modules/init/profile.sh
30 changes: 1 addition & 29 deletions ci-scripts/dockerfiles/Tmod4.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,39 +2,11 @@
# Execute this from the top-level ReFrame source directory
#

FROM ubuntu:20.04

ENV TZ=Europe/Zurich
ENV DEBIAN_FRONTEND=noninteractive
ENV _TMOD_VER=4.6.0
FROM reframehpc/rfm-ci-base:tmod4

# ReFrame user
RUN useradd -ms /bin/bash rfmuser

# ReFrame requirements
RUN \
apt-get -y update && \
apt-get -y install ca-certificates && \
update-ca-certificates && \
apt-get -y install gcc && \
apt-get -y install make && \
apt-get -y install git && \
apt-get -y install python3 python3-pip

# Required utilities
RUN apt-get -y install wget

# Install Tmod4
RUN \
apt-get -y install autoconf && \
apt-get -y install tcl-dev && \
wget -q https://github.com/cea-hpc/modules/archive/v${_TMOD_VER}.tar.gz -O tmod.tar.gz && \
tar xzf tmod.tar.gz && \
cd modules-${_TMOD_VER} && \
./configure && make install

ENV BASH_ENV=/usr/local/Modules/init/profile.sh

USER rfmuser

# Install ReFrame from the current directory
Expand Down