Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add single-user scripts to minimal-notebook. #185

Merged
merged 3 commits into from Apr 14, 2016

Conversation

@jtyberg
Copy link
Member

jtyberg commented Apr 12, 2016

Here is what I think is a light-touch approach to enabling the *-notebook stacks to run the notebook servers in different modes.

  1. Include the single-user scripts in the base minimal-notebook image to make them available to all notebook stacks.
  2. Run the notebook server in single-user mode by overriding the Docker run CMD. For example, when spawning from JupyterHub, configure the DockerSpawner as follows:
# Spawn user containers from this image
c.DockerSpawner.container_image = 'jupyter/all-spark-notebook'

# Have the Spawner override the Docker run command
c.DockerSpawner.extra_create_kwargs.update({ 
    'command': '/usr/local/bin/singleuser.sh'
})

As a consumer of these images, I prefer this as opposed to dealing with a multiple images per stack, but that's just my opinion.

README.md Outdated
@@ -43,6 +41,7 @@ Starting with [git commit SHA 9bd33dcc8688](https://github.com/jupyter/docker-st
## Other Tips

* `tini -- start-notebook.sh` is the default Docker entrypoint-plus-command in every notebook stack. If you plan to modify it any way, be sure to check the *Notebook Options* section of your stack's README to understand the consequences.
* Every notebook stack also includes an alternate script, [singleuser.sh](https://raw.githubusercontent.com/jupyter/dockerspawner/master/singleuser/singleuser.sh), which you can use as the command to run the Notebook server in single-user mode, as required by [JupyterHub](https://jupyterhub.readthedocs.org).

This comment has been minimized.

Copy link
@parente

parente Apr 12, 2016

Member

Does singleuser.sh also run under tini?

This comment has been minimized.

Copy link
@jtyberg

jtyberg Apr 12, 2016

Author Member

Yes. By using the Docker mechanism to override the run CMD, we still run under the same entrypoint.

docker inspect jupyter-jtyberg
...
            "Cmd": [
                "/usr/local/bin/singleuser.sh"
            ],
            "Entrypoint": [
                "tini",
                "--"
            ],

Perhaps I need to update the wording to make this explicit.

Now, if you change the entrypoint, you better know what you're doing.

@@ -243,11 +244,16 @@ source deactivate
The commands `ipython`, `python`, `pip`, `easy_install`, and `conda` (among others) are available in both environments.


## JupyterHub
## JupyterHub <a id="JupyterHub"></a>

This comment has been minimized.

Copy link
@parente

parente Apr 12, 2016

Member

Pretty sure you can just link to #JupyterHub in the git blob view without the <a>

This comment has been minimized.

Copy link
@parente

parente Apr 12, 2016

Member

Repeat for all below.

# Fetch additional scripts that run Notebook server in single-user mode (for
# use with JupyterHub)
RUN wget \
-q https://raw.githubusercontent.com/jupyter/jupyterhub/master/scripts/jupyterhub-singleuser \

This comment has been minimized.

Copy link
@parente

parente Apr 12, 2016

Member

If you point to master, there's no way to know exactly what version is getting put into a particular docker-stacks build. Nor is there a way for us to update via PR. We just have to manually force it. Suggest pointing to a stable tag (preferred) or a git SHA.

This comment has been minimized.

Copy link
@minrk

minrk Apr 12, 2016

Member

This is the only piece that makes me less confident about this PR being the way to go. JupyterHub isn't sensitive to the versions of anything in these images other than the version of JupyterHub whence came this singleuser script, which really should match the parent JupyterHub.

Pinning to stable (0.5.0) makes sense, though.

@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 12, 2016

If this goes in, we also need to update the docker hierarchy diagram in the README to remove all the singleuser notebook boxes. The source for regenerating the svg at http://interactive.blockdiag.com/ is in the README.

@jtyberg

This comment has been minimized.

Copy link
Member Author

jtyberg commented Apr 12, 2016

Assuming we're going to remove the build/%-singleuser builds, then yes, I will update the diagram too. I'm just not sure how extensively they're used downstream at this point (by DockerSpawner). Most of the *-singleuser stacks have about 30-40 pulls from DockerHub since being introduced.

@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 12, 2016

I'm just not sure how extensively they're used downstream

They're really really new. And we won't delete the images that we pushed. We'll leave them there for some time and mark them deprecated on their Docker Hub pages with pointers to the new way of doing things.

@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 12, 2016

In the same vein, if this is the path we go down, we need to remove the .sh and automation in the makefile for building / pushing the singleuser images. But I think @minrk should weigh in on the change before we go there.

@minrk

This comment has been minimized.

Copy link
Member

minrk commented Apr 12, 2016

I think this is great. The only thing I'm not sure about going forward is matching JupyterHub and jupyterhub-singleuser script versions.

What I would really like is if DockerSpawner added the singleuser script to the container at runtime, but I'm not sure there's a good way to do that. If that were the case, there would no longer be a reason to have any singleuser stuff in this repo.

@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 12, 2016

What I would really like is if DockerSpawner added the singleuser script to the container at runtime, but I'm not sure there's a good way to do that. If that were the case, there would no longer be a reason to have any singleuser stuff in this repo.

The ways I've seen this done is either by a volume mounting the file (doesn't work distributed), by pushing content into the container on stdin to a process that knows what to do with it (e.g., bash script that takes it, writes it to disk, execs it), and by having a process in the container pull the content from a URL (e.g., script that wgets/curls/requests a file from a HTTP server and execs it). I've had the most success with the last one. Would there be a way in JupyterHub for one of the Hub processes to host the singleuser script for user containers to fetch on startup?

EDIT: To be clear, I think these are ideas that can be tried later. I'm perfectly fine with the approach in this PR for now.

@jtyberg

This comment has been minimized.

Copy link
Member Author

jtyberg commented Apr 12, 2016

I'm in agreement that this does not feel quite right. Truthfully, I don't like reaching across two repos to grab startup scripts, especially if JupyterHub startup is a moving target.

Will look into a way to seed the single-user containers with a startup script.

@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 12, 2016

How about:

  1. conda install -y jupyterhub==0.5* in minimal-notebook JUST to get the jupyterhub-singleuser script
  2. Create a start-singleuser.sh script that lives in the minimal-notebook folder alongside start-notebook.sh
  3. Add to the README that the images are compatible with JupyterHub 0.5.
  4. When someone asks to make these images compatible with a new version of Jupyterhub, the PR must:
    a. bump the conda/pip install of jupyterhub to the desired version
    b. make sure start-singleuser.sh still invokes jupyter-singleuser properly (e.g., CLI hasn't changed)

With this setup, it's no different than bumping the version of jupyter notebook in the container and making sure start-notebook.sh still works with it.

I like consistency.

@jtyberg jtyberg force-pushed the jtyberg:single-user branch 2 times, most recently from 6368433 to c1b5431 Apr 12, 2016
@jtyberg

This comment has been minimized.

Copy link
Member Author

jtyberg commented Apr 12, 2016

So bootstrapping spawned containers by pulling the jupyterhub-singleuser script from JupyterHub at runtime feels a bit weird (it reminds me of the way Spark works, so maybe that's what makes me uneasy).

For now, I'm in favor of pinning the version of the script. Pinning known releases is what we've been doing all along in these stacks. I used pip install jupyterhub==0.5* as @parente suggested.

@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 12, 2016

LGTM as a fine starting point. The start-singleuser.sh can be extended over time in conjunction with feature requests here and version bumps to JupyterHub just like any of the other packages in the docker-stacks.

Of course, I don't have a JHub setup to test it. @jtyberg I'm assuming you did. Does anyone else want to try before we merge? We can always iterate as we find issues.

/cc @carolynvs (who I should have thought to ping earlier)

jtyberg added 3 commits Apr 11, 2016
stacks easily usable with JupyterHub.

* pip install jupyterhub to gain access to the jupyterhub-singleuser
  startup script, which starts a single-user instance of the Notebook
  server
* Add shell script to wrap jupyterhub-singleuser script; use as
  alternate Docker command

fixes #181

(c) Copyright IBM Corp. 2016
Remove additional *-singleuser image builds in favor of including the
single-user startup scripts in the *-notebook images.

(c) Copyright IBM Corp. 2016
Remove singleuser images.

(c) Copyright IBM Corp. 2016
@jtyberg jtyberg force-pushed the jtyberg:single-user branch from 4a56bbd to 34da9f0 Apr 12, 2016
@jtyberg

This comment has been minimized.

Copy link
Member Author

jtyberg commented Apr 14, 2016

FWIW, I've tested this after building multiple *-notebook stacks locally, and spawning them in a JupyterHub deployment (using DockerSpawner, of course).

@parente parente merged commit 2d878db into jupyter:master Apr 14, 2016
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 14, 2016

Building now. After the images are on dockerhub:

  • Mark the singleuser images as deprecated on DH
  • Update the wiki recipe here about JHub
@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 14, 2016

Tag 2d878db pushed to DH. Wiki page updated. Singleuser images marked deprecated on DH.

@minrk

This comment has been minimized.

Copy link
Member

minrk commented Apr 14, 2016

Thanks!

@jtyberg

This comment has been minimized.

Copy link
Member Author

jtyberg commented Apr 14, 2016

Thanks, @parente. I've pulled the images and tested in my local JupyterHub environment. 👍

@parente

This comment has been minimized.

Copy link
Member

parente commented Apr 14, 2016

🍰 for @jtyberg, you did all the hard work!

@parente parente referenced this pull request May 18, 2016
rochaporto pushed a commit to rochaporto/docker-stacks that referenced this pull request Jan 23, 2019
update log.warn (deprecated) to log.warning
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.