Add conda and pip packages at run time

[mrocklin]

See #4 and helm/charts#2912

cc @danielfrg also @ogrisel

[mrocklin]

I've also changed the Jupyter Dockerfile to install dask directly into the root environment used by the Jupyter notebook server by default.

[mrocklin]

I had a bit of trouble building the jupyter docker image because of its reliance on base/prepare.sh. @danielfrg how did you build this?

If I copy over the prepare.sh script explicitly then I can do the following and everything works as expected

cd base
docker build -t dask-foo .

cd ../notebook
docker build -t dask-foo-notebook .

docker run --network=host dask-foo dask-scheduler
docker run --network=host -e EXTRA_PIP_PACKAGES="s3fs gcsfs" dask-foo dask-worker localhost:8786
docker run --network=host -e EXTRA_PIP_PACKAGES="s3fs gcsfs" dask-foo dask-worker localhost:8786
docker run --network=host -e EXTRA_PIP_PACKAGES="s3fs gcsfs" dask-foo-notebook

[danielfrg]

I used docker-compose build from the root of the repo to build the images. This will make it use the right docker context for each image.

[danielfrg]

Yes, this is definitely the solution to free the CMD that you mention in #4 i didn't do this initially because it would make it impossible to skip the prepare.sh but in general its not a big issue i think.

[danielfrg]

👍

[mrocklin]

What's your publication mechanism? Just build locally and push to dockerhub?

[danielfrg]

What's your publication mechanism? Just build locally and push to dockerhub?

Yes

[danielfrg]

Actually nevermind, I did setup up the two images on dockerhub so they will get automatically build from new commits on master.

[mrocklin]

Grand. OK to merge?

[danielfrg]

yes. lgtm

[mrocklin]

Thank you for your help @danielfrg !

…

On Sat, Dec 2, 2017 at 9:27 PM, Daniel Rodriguez ***@***.***> wrote: yes. lgtm — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszB67LKKqEJHFMbDaKFzeq1j8LF9Xks5s8gcIgaJpZM4QzcJc> .