Skip to content

ORC-694: Update docker files for adding Java11 support #655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yuokada wants to merge 9 commits into from
Closed

ORC-694: Update docker files for adding Java11 support #655

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
89a2cb8
Improve Dockerfile per os to support jdk argument
yuokada Mar 15, 2021
e9d4f5b
Update shell scripts to support docker images for JDK 11
yuokada Mar 15, 2021
a85bd62
Update README.md for docker
yuokada Mar 15, 2021
3475e65
fixup! Improve Dockerfile per os to support jdk argument
yuokada Mar 16, 2021
28dbb4d
Improve Dockerfile for centos7/centos8
yuokada Mar 17, 2021
e08d96e
Improve message in run-all.sh
yuokada Mar 17, 2021
f7329c0
Update README
yuokada Mar 17, 2021
27a8570
Skip the building operations with specific combinations
yuokada Mar 18, 2021
415a3bd
Update README.md
yuokada Mar 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,17 @@ The scripts are:
* `run-one.sh` *owner* *branch* *os* - test the owner's branch on one OS
* `reinit.sh` - rebuild all of the base images without the image cache

`run-all.sh`, `run-one.sh` and `reinit.sh` tests both on jdk8 and 11 across OSes

A base image for each OS is built using:

@pgaref pgaref Mar 17, 2021 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add above that:
run-all.sh, run-one.sh and reinit.sh tests both on jdk8 and 11 across OSes


cd docker/$os
docker build -t orc-$os .
FOR jdk8: docker build -t "orc-$os-jdk8" --build-arg jdk=8 .
FOR jdk11: docker build -t "orc-$os-jdk11" --build-arg jdk=11 .

## JDK 11 support

For debian9 & ubuntu16: JDK 11 support isn't provided.

## Clean up

Expand Down
18 changes: 15 additions & 3 deletions docker/centos7/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

FROM centos:7
LABEL maintainer="Apache ORC project <dev@orc.apache.org>"
ARG jdk=8

RUN yum check-update || true
RUN yum -y install epel-release
Expand All @@ -31,16 +32,27 @@ RUN yum install -y \
gcc-c++ \
gettext-devel \
git \
java-1.8.0-openjdk \
java-1.8.0-openjdk-devel \
libtool \
make \
maven \
openssl-devel \
tar \
wget \
which \
zlib-devel
zlib-devel && \
if [ "${jdk}" = "11" ] ; then \
yum install -y \
java-11-openjdk \
java-11-openjdk-devel \
&& \
update-alternatives --set java $(readlink -f /usr/lib/jvm/java-11-openjdk/bin/java) && \
update-alternatives --set javac $(readlink -f /usr/lib/jvm/java-11-openjdk/bin/javac) \
; else \
yum install -y \
java-1.8.0-openjdk \
java-1.8.0-openjdk-devel \
; fi

# Our scripts assume that cmake is called 'cmake'
RUN ln -s /usr/bin/cmake3 /usr/bin/cmake

Expand Down
17 changes: 14 additions & 3 deletions docker/centos8/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

FROM centos:8
LABEL maintainer="Apache ORC project <dev@orc.apache.org>"
ARG jdk=8

RUN yum check-update || true
RUN yum install -y \
Expand All @@ -30,16 +31,26 @@ RUN yum install -y \
gcc-c++ \
gettext-devel \
git \
java-1.8.0-openjdk \
java-1.8.0-openjdk-devel \
libtool \
make \
maven \
openssl-devel \
tar \
wget \
which \
zlib-devel
zlib-devel && \
if [ "${jdk}" = "11" ] ; then \
yum install -y \
java-11-openjdk \
java-11-openjdk-devel \
&& \
update-alternatives --set java $(readlink -f /usr/lib/jvm/java-11-openjdk/bin/java) && \
update-alternatives --set javac $(readlink -f /usr/lib/jvm/java-11-openjdk/bin/javac) \
; else \
yum install -y \
java-1.8.0-openjdk \
java-1.8.0-openjdk-devel \
; fi

ENV TZ=America/Los_Angeles
WORKDIR /root
Expand Down
12 changes: 9 additions & 3 deletions docker/reinit.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,15 @@
# limitations under the License.

start=`date`
for os in `cat os-list.txt`; do
echo "Re-initialize $os"
( cd $os && docker build --no-cache -t "orc-$os" . )
for jdk in 8 11; do
for os in `cat os-list.txt`; do
if [[ "$os" = "debian10" && "$jdk" = "8" ]] || [[ "$os" = "debian9" && "$jdk" = "11" ]] || [[ "$os" = "ubuntu16" && "$jdk" = "11" ]]; then
echo "Skip an initialize $os with $jdk"
continue
fi
echo "Re-initialize $os with $jdk"
( cd $os && docker build --no-cache -t "orc-${os}-jdk${jdk}" --build-arg jdk=${jdk} . )
done
done
echo "Start: $start"
echo "End:" `date`
12 changes: 9 additions & 3 deletions docker/run-all.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,15 @@ function failure {
rm -f logs/pids.txt logs/*.log

start=`date`
for os in `cat os-list.txt`; do
echo "Building $os"
( cd $os && docker build -t "orc-$os" . ) > logs/$os-build.log 2>&1 || exit 1
for jdk in 8 11; do
for os in `cat os-list.txt`; do
if [[ "$os" = "debian10" && "$jdk" = "8" ]] || [[ "$os" = "debian9" && "$jdk" = "11" ]] || [[ "$os" = "ubuntu16" && "$jdk" = "11" ]]; then
echo "Skip building $os with $jdk"
continue
fi
echo "Building $os for $jdk"
( cd $os && docker build -t "orc-$os-jdk${jdk}" --build-arg jdk=$jdk . ) > logs/${os}-jdk${jdk}-build.log 2>&1 || exit 1
done
done
testStart=`date`

Expand Down
12 changes: 9 additions & 3 deletions docker/run-one.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,14 @@ else
OPTS=""
;;
esac
docker run $VOLUME "orc-$OS" /bin/bash -c \
"$CLONE && $MAKEDIR && cmake $OPTS .. && make package test-out" \
|| failure

for jdk in 8 11; do
if [[ "$OS" = "debian10" && "$jdk" = "8" ]] || [[ "$OS" = "debian9" && "$jdk" = "11" ]] || [[ "$OS" = "ubuntu16" && "$jdk" = "11" ]]; then
continue
fi
docker run $VOLUME "orc-$OS-jdk${jdk}" /bin/bash -c \
"$CLONE && $MAKEDIR && cmake $OPTS .. && make package test-out" \
|| failure
done
fi
echo "Finished $OS at $(date)"
5 changes: 3 additions & 2 deletions docker/ubuntu18/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

FROM ubuntu:18.04
LABEL maintainer="Apache ORC project <dev@orc.apache.org>"
ARG jdk=8

RUN ln -fs /usr/share/zoneinfo/America/Los_Angeles /etc/localtime
RUN apt-get update
Expand All @@ -32,9 +33,9 @@ RUN apt-get install -y \
make \
curl \
maven \
openjdk-8-jdk \
openjdk-${jdk}-jdk \
tzdata
RUN update-java-alternatives --set java-1.8.0-openjdk-amd64
RUN update-java-alternatives --set java-1.${jdk}.0-openjdk-amd64
Comment thread
dongjoon-hyun marked this conversation as resolved.

WORKDIR /root
VOLUME /root/.m2/repository
Expand Down
5 changes: 3 additions & 2 deletions docker/ubuntu20-clang/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

FROM ubuntu:20.04
LABEL maintainer="Apache ORC project <dev@orc.apache.org>"
ARG jdk=8

RUN ln -fs /usr/share/zoneinfo/America/Los_Angeles /etc/localtime
RUN apt-get update
Expand All @@ -31,9 +32,9 @@ RUN apt-get install -y \
make \
curl \
maven \
openjdk-8-jdk \
openjdk-${jdk}-jdk \
tzdata
RUN update-java-alternatives --set java-1.8.0-openjdk-amd64
RUN update-java-alternatives --set java-1.${jdk}.0-openjdk-amd64
Comment thread
dongjoon-hyun marked this conversation as resolved.

ENV CC=clang
ENV CXX=clang++
Expand Down
5 changes: 3 additions & 2 deletions docker/ubuntu20/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

FROM ubuntu:20.04
LABEL maintainer="Apache ORC project <dev@orc.apache.org>"
ARG jdk=8

RUN ln -fs /usr/share/zoneinfo/America/Los_Angeles /etc/localtime
RUN apt-get update
Expand All @@ -32,9 +33,9 @@ RUN apt-get install -y \
make \
curl \
maven \
openjdk-8-jdk \
openjdk-${jdk}-jdk \
tzdata
RUN update-java-alternatives --set java-1.8.0-openjdk-amd64
RUN update-java-alternatives --set java-1.${jdk}.0-openjdk-amd64
Comment thread
dongjoon-hyun marked this conversation as resolved.

WORKDIR /root
VOLUME /root/.m2/repository
Expand Down