Skip to content

Commit

Permalink
ARROW-2350: Consolidated RUN step in spark_integration Dockerfile
Browse files Browse the repository at this point in the history
I am a big fan of this project! In this PR, I propose a small change to the `spark_integration` Dockerfile. See [ARROW-2350](https://issues.apache.org/jira/browse/ARROW-2350) for a full description of what I'm proposing here.

This change cuts the size of the `spark_integration` container from **2.65 GB** to **2.26 GB**.

Author: James Lamb <james.lamb@uptake.com>

Closes #1787 from jameslamb/smaller_docker and squashes the following commits:

7343d8d <James Lamb> ARROW-2350: Consolidated RUN step in spark_integration Dockerfile
  • Loading branch information
James Lamb authored and xhochy committed Mar 25, 2018
1 parent af6e3ec commit 29f744f
Showing 1 changed file with 18 additions and 16 deletions.
34 changes: 18 additions & 16 deletions dev/spark_integration/Dockerfile
Expand Up @@ -17,24 +17,23 @@
FROM maven:3.5.2-jdk-8-slim

# Basic OS utilities
RUN apt-get update && apt-get install -y \
RUN apt-get update && \
apt-get install -y \
wget \
git build-essential \
software-properties-common

# This will install conda in /home/ubuntu/miniconda
RUN wget -O /tmp/miniconda.sh \
git \
build-essential \
software-properties-common && \
apt-get clean && \
# install conda in /home/ubuntu/miniconda
wget -O /tmp/miniconda.sh \
https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
bash /tmp/miniconda.sh -b -p /home/ubuntu/miniconda && \
rm /tmp/miniconda.sh

# Python dependencies
RUN apt-get install -y \
pkg-config

# Create Conda environment
ENV PATH="/home/ubuntu/miniconda/bin:${PATH}"
RUN conda create -y -q -n pyarrow-dev \
rm /tmp/miniconda.sh && \
# Python dependencies
apt-get install -y \
pkg-config && \
# Create Conda environment
/home/ubuntu/miniconda/bin/conda create -y -q -n pyarrow-dev \
# Python
python=2.7 \
numpy \
Expand All @@ -59,7 +58,10 @@ RUN conda create -y -q -n pyarrow-dev \
jemalloc \
lz4-c \
zstd \
-c conda-forge
-c conda-forge && \
/home/ubuntu/miniconda/bin/conda clean --all

ENV PATH="/home/ubuntu/miniconda/bin:${PATH}"

ADD . /apache-arrow
WORKDIR /apache-arrow
Expand Down

0 comments on commit 29f744f

Please sign in to comment.