Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-2350: Consolidated RUN step in spark_integration Dockerfile #1787

Closed
wants to merge 1 commit into from
Closed

ARROW-2350: Consolidated RUN step in spark_integration Dockerfile #1787

wants to merge 1 commit into from

Conversation

jameslamb
Copy link
Contributor

I am a big fan of this project! In this PR, I propose a small change to the spark_integration Dockerfile. See ARROW-2350 for a full description of what I'm proposing here.

This change cuts the size of the spark_integration container from 2.65 GB to 2.26 GB.

Copy link
Member

@xhochy xhochy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thanks for having a look at this.

We might want to put this in future into a single bash script to improve readability.

@xhochy xhochy closed this in 29f744f Mar 25, 2018
@jameslamb jameslamb deleted the smaller_docker branch March 25, 2018 15:48
@jameslamb
Copy link
Contributor Author

@xhochy All the dockerfiles in https://github.com/apache/arrow/tree/master/dev could use this attention. Would you welcome a PR making similar organizational changes to them?

@xhochy
Copy link
Member

xhochy commented Mar 28, 2018

@jameslamb yes, that would be great. It looks like we could trim down all of them. In some cases, we install python packages from pip install git+…. These steps should maybe stay separate so that we can only delete the docker cache for them and have a fast rebuild.

@jameslamb
Copy link
Contributor Author

makes sense to me! I will probably submit a PR over the weekend.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants