Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Environment setup before starting the kernel #847

Closed
rbiswas4 opened this issue Nov 7, 2016 · 8 comments
Closed

Environment setup before starting the kernel #847

rbiswas4 opened this issue Nov 7, 2016 · 8 comments

Comments

@rbiswas4
Copy link

rbiswas4 commented Nov 7, 2016

We are trying to use the JupyterHub setup on nersc, but we need to setup our own code (LSST stack) by running a couple of setup scripts (which work on the NERSC machines) before starting an ipython/jupyter notebook, so that our package is setup. How can we achieve this?

@yuvipanda
Copy link
Contributor

Hello!

What spawner are you using? You can probably use c.Spawner.cmd to do what you want. This is the command that is run by JupyterHub spawners. You can override that to be another command that you provision which does whatever setup you want and then calls the jupyterhub-singleuser command (which is the default). This will all be run as the user who is being spawned.

@rbiswas4
Copy link
Author

Hi,

Sorry, I was not clear. I was not using a spawner explicitly, I was simply trying to point the browser at nersc.jupyter.gov which already spawns a server internally (I think). I would be happy to spawn something myself, if that helps.

@minrk
Copy link
Member

minrk commented Nov 12, 2016

Do you mean before starting a single notebook, or before starting your whole notebook server?

Depending on what the commands are and how early they need to run, perhaps the simplest thing to do is to put a Python script in ~/.ipython/profile_default/startup/init_lsst.py, which will be executed by IPython early in its setup, but after initializing IPython.

If it needs to be shell commands executed before starting Python, perhaps the thing to do is to create a kernelspec for your setup. You can start by creating an lsst kernelspec starting with IPython's default:

ipython kernel install --user --name lsst

which will create a directory like:

~/.local/share/jupyter/kernels/lsst

containing a kernel.json

You can then write a script (lsst-kernel.sh) that does your setup followed by the IPython kernel launch:

#!/usr/bin/env bash
module lsst
# setup lsst, environment variables, etc.

# this is the critical part, and should be at the end of your script:
exec python -m ipykernel $@

Finally, you can modify the ~/.local/share/jupyter/kernels/lsst/kernel.json file to call your script, changing:

 "argv": [
  "/usr/bin/python",
  "-m",
  "ipykernel",
  "-f",
  "{connection_file}"
 ],

to

 "argv": [
  "/path/to/lsst-kernel.sh",
  "-f",
  "{connection_file}"
 ],

(replacing python -m ipykernel with /path/to/lsst-kernel.sh).

@jbkalmbach
Copy link

jbkalmbach commented Dec 9, 2016

Hi @minrk I am picking up on this for @rbiswas4 and whenever I try to get this to run the kernel says it is loading and then says it is ready, but as soon as that happens I get a dead kernel. Any idea why?

@minrk
Copy link
Member

minrk commented Dec 14, 2016

Thanks for picking it up! Can you see the logs for the single-user server? If the first execution dies right away, it's likely that there's a low-level error that wouldn't propagate all the way to the client.

@jbkalmbach
Copy link

Hi @minrk it was actually some of what was going on behind the scenes at NERSC that wouldn't let us execute the commands in the shell script we wanted. We got in touch with them and sorted it out and now your steps from November 12 work perfectly. Thanks for your help!

@minrk
Copy link
Member

minrk commented Dec 15, 2016

Great to hear! Closing then.

@GrantFinneman
Copy link

I understand this is closed but I am running into the same exact issue but using The Littlest Jupyterhub server instead of a single user install. My kernel.json is different but I need the same thing. I have a couple compiled programs that have libraries sourced by my .bashrc in the jupyter account but since the notebooks are spawned by the default spawner they do not have the same environment variables. How would I accomplish this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants