New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environment setup before starting the kernel #847

Closed
rbiswas4 opened this Issue Nov 7, 2016 · 7 comments

Comments

Projects
None yet
4 participants
@rbiswas4

rbiswas4 commented Nov 7, 2016

We are trying to use the JupyterHub setup on nersc, but we need to setup our own code (LSST stack) by running a couple of setup scripts (which work on the NERSC machines) before starting an ipython/jupyter notebook, so that our package is setup. How can we achieve this?

@yuvipanda

This comment has been minimized.

Member

yuvipanda commented Nov 8, 2016

Hello!

What spawner are you using? You can probably use c.Spawner.cmd to do what you want. This is the command that is run by JupyterHub spawners. You can override that to be another command that you provision which does whatever setup you want and then calls the jupyterhub-singleuser command (which is the default). This will all be run as the user who is being spawned.

@rbiswas4

This comment has been minimized.

rbiswas4 commented Nov 10, 2016

Hi,

Sorry, I was not clear. I was not using a spawner explicitly, I was simply trying to point the browser at nersc.jupyter.gov which already spawns a server internally (I think). I would be happy to spawn something myself, if that helps.

@minrk

This comment has been minimized.

Member

minrk commented Nov 12, 2016

Do you mean before starting a single notebook, or before starting your whole notebook server?

Depending on what the commands are and how early they need to run, perhaps the simplest thing to do is to put a Python script in ~/.ipython/profile_default/startup/init_lsst.py, which will be executed by IPython early in its setup, but after initializing IPython.

If it needs to be shell commands executed before starting Python, perhaps the thing to do is to create a kernelspec for your setup. You can start by creating an lsst kernelspec starting with IPython's default:

ipython kernel install --user --name lsst

which will create a directory like:

~/.local/share/jupyter/kernels/lsst

containing a kernel.json

You can then write a script (lsst-kernel.sh) that does your setup followed by the IPython kernel launch:

#!/usr/bin/env bash
module lsst
# setup lsst, environment variables, etc.

# this is the critical part, and should be at the end of your script:
exec python -m ipykernel $@

Finally, you can modify the ~/.local/share/jupyter/kernels/lsst/kernel.json file to call your script, changing:

 "argv": [
  "/usr/bin/python",
  "-m",
  "ipykernel",
  "-f",
  "{connection_file}"
 ],

to

 "argv": [
  "/path/to/lsst-kernel.sh",
  "-f",
  "{connection_file}"
 ],

(replacing python -m ipykernel with /path/to/lsst-kernel.sh).

@jbkalmbach

This comment has been minimized.

jbkalmbach commented Dec 9, 2016

Hi @minrk I am picking up on this for @rbiswas4 and whenever I try to get this to run the kernel says it is loading and then says it is ready, but as soon as that happens I get a dead kernel. Any idea why?

@minrk

This comment has been minimized.

Member

minrk commented Dec 14, 2016

Thanks for picking it up! Can you see the logs for the single-user server? If the first execution dies right away, it's likely that there's a low-level error that wouldn't propagate all the way to the client.

@jbkalmbach

This comment has been minimized.

jbkalmbach commented Dec 14, 2016

Hi @minrk it was actually some of what was going on behind the scenes at NERSC that wouldn't let us execute the commands in the shell script we wanted. We got in touch with them and sorted it out and now your steps from November 12 work perfectly. Thanks for your help!

@minrk

This comment has been minimized.

Member

minrk commented Dec 15, 2016

Great to hear! Closing then.

@minrk minrk closed this Dec 15, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment