Adds a Dockerfile for docker build of coLaboratory #92
Conversation
|
Hi @KesterTong, this looks great. Any chance you could add in "yt" as well, for parity with the PNaCl kernel? |
| pip install pyzmq jinja2 tornado && \ | ||
| pip install bokeh && \ | ||
| pip install ipython[notebook] && \ | ||
| git clone -b master --recursive https://github.com/jupyter/colaboratory.git && \ |
There was a problem hiding this comment.
Why not ADD the colaboratory stuff directly? Is this to workaround submodules?
There was a problem hiding this comment.
So far I had used a dockerfile that wasn't part of this project, and I was just committing it here, so I hadn't thought of this possibility.
|
The Docker hub will also take any README.md that sits in the directory this Dockerfile is at to populate the information page. |
|
@rgbkrk I think I don't fully understand the relationship between github repos and Docker. So far I was treating them as completely distinct, but this seems to be suboptimal. Is there any particular setup you recommend, e.g. using Docker's automated build system? |
|
@matthewturk I'm not clear on how to install yt and its Python bindings. Can you give me the command line instructions to do this, and I will add them to the Dockerfile |
|
Sure. pip install yt On Fri, Aug 15, 2014 at 3:58 PM, KesterTong notifications@github.com
|
|
@matthewturk Too easy |
|
@rgbkrk I just set up an automated Docker build at https://registry.hub.docker.com/u/kestert/colaboratory-test/ I'm interested to see how long building this ~1GB image takes. |
The Docker hub will clone the entire repository then work from the directory this Dockerfile is specified in:
If using ADD, it will add the entire directory to the Docker image itself. |
Forever. Hopefully soon I'll make a post about how to create a completely awkward workflow with Docker + the scipy stack in the middle of contest, win the contest, then proceed to be disqualified for turning it in late. Once we have base images we can pull (and are not iterating on them), others can benefit from just having to pull the images down. How well will this work if used on a remote server? |
|
@rgbkrk, that gives me the idea that it would be better to use an IPython base image, and add the few extra libraries we need. Is there an official IPython base image that is close to what we have here? It doesn't surprise me that the automated build process is slow. It takes about 20 minutes for me to build on an n1-standard-1 GCE instance, which costs 7c per hour. These costs would add up pretty quick given that it rebuilds on every push to the repo. Were you asking how well our docker image would work on a remote server? The main issue is there is no authentication (and it listens on every port!) so it's only designed for one use case: servers that users connect to over ssh with port forwarding. But I hope that we can collaborate with people who want to use this image to run a real webserver. The authentication just isn't there yet. |
|
The current base image lives at https://github.com/ipython/docker-notebook. More than happy to collab on that one. I made some choices there that others may not agree with (notebooks in /notebooks, SSL created by default, password required). I'm willing to dial that back though. I've been looking forward to working on that image and derivative images after starting a lively thread on the IPython-dev mailing list and actually seeing what the workflow would be like (especially under pressure). |
Cython is required to build yt
|
@rgbkrk actually the build went surprisingly fast, it got all the way up to yt before breaking (added a fix for that). I'm quite surprised since these resources aren't cheap, but I guess their internal costs are much lower. I will take a look at your build. It would probably be more efficient to work together on a base image, but given how quickly it just built, having a separate dockerfile here will work too. |
|
@rgbkrk I just took a look at your docker build and some of the discussion in python-dev. My own view is that it's good to separate the libraries from the image that runs the server, so that people can create their own custom servers (which seems to be popular with IPython). Also, the issues are very different. For libraries, the concern is minimizing image size (and possibly making sure package versions are compatible). For the server, the concern is providing sensible defaults. So I would suggest the following four builds (with dependency graph 1 -> 2, 1-> 3, 2 -> 4)
I will also take a look at the server setup, that is something that would be good to have in common between IPython and coLab if possible. |
|
@KesterTong Completely agree with the dependency graph you have outlined. That's where my current vote is, as is using plain pip over conda (also a discussion point on the mailing list). |
|
@rgbkrk I will read more of that thread and add my opinion if it's relevant. I also agreed the reasons you and others gave for preferring pip over conda. Can you merge this PR if you're happy with the current state. I just tested the automated build at https://github.com/ipython/docker-notebook and it works. I'm interested in reading more about your experience because so far the automated build has been going very smoothly. Because I had to move the Dockerfile to the base of the repo, the automated build uses the README.md at the root of this repo. |
|
Gladly, I'm really happy with this. I'll set up the automated build for jupyter/colaboratory too. |
Adds a Dockerfile for docker build of coLaboratory
Adds a Dockerfile to build a docker image for coLaboratory. This image is designed to run on a server that users connect to through ssh using port forwarding. It is not intended to act as a web server, as there is no authentication done by the kernel.