$PATH in jupyterhub notebook does not reflect conda env path

[johnchase]

Hello, I have been using jupyter for awhile and appreciate the work that has gone into it. I am running into an issue that I have not been able to find a solution for and was hoping to get some help. The issue I am running into is that the path variable in the notebooks does not contain the path to the conda environment (This is actully not surprising but I would like to change it if possible)

I am running jupyterhub following the zero-to-jupyterhub-with-kubernetes tutorial very closely. I would like to allow users to create their own conda environments and have populated the users home directory with a .condarc file:

$ cat .condarc
envs_dirs:
  - /home/jovyan/my-conda-envs/

I can create a new environment from the terminal, activate it, and view the path:

conda create -n test nb_conda \
activate environment \
source activate environment test \
echo $PATH \
/home/jovyan/my-conda-envs/test/bin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

This is what I am hoping to see. However when I navigate to the jupyter server and start a notebook with the conda environment I created and the run:

import os
print(os.environ['PATH'])
/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

The path does not point to my conda environment. As I mentioned this is not entirely unexpected as I realize jupyterhub is being served from a different environment, however, I am wondering if it is possible to make the environment in the notebook reflect the terminal. At a high level I am trying to use a python package that makes makes command line calls to other resources such as .r files, which are located in the conda environment I created. Thanks for your help, I believe this may be related to #2091 .

[jhermann]

Consider using the Spawner.pre_spawn_hook to manipulate the environment. Since you want user specific customization, the hub's conda environment is likely the wrong one to use.

That means the notebook server should possibly be launched using the user's env, or maybe some PYTHONPATH manipulation is the right thing to do.

My goal is to eventually integrate a user's "--user" installed packages into the spawned notebooks and default kernel (using the hub's env – in my case a virtualenv)

[jhermann]

Some quick testing… running the following in a notebook makes user packages available:

import os, sys
sys.path.insert(1, 
    os.path.expanduser('~/.local/lib/python{0}.{1}/site-packages'
                       .format(*sys.version.split('.'), 2)))
sys.version, sys.version_info, sys.path

Beforehand, you need to install the user packages on a shell prompt (use your hub's Python interpreter / version):

python3.6 -m pip install --user -UI pytz pyparsing seaborn

Putting the sys.path manipulation into the pre_spawn_hook might be all that needs to be done to make this a default.

[johnchase]

Thanks @jhermann , this is similar to what I was doing which was modifying the path like so from within a notebook:

import os
os.environ['PATH'] = '/home/jovyan/my-conda-envs/test/bin:' + os.environ['PATH']

IIUC Your solution would make available any packages that were installed globally it site-packages and so you could add the code. The python package I am using is installing the non-python code into the conda environment itself. I am still wrapping my head around much of this so I apologize if I am misunderstanding.

[jhermann]

You likely need to start the notebook server using the user's conda env (and that one needs "notebook" installed). Technically that should be rather straightforward and if not possible by config, then certainly by writing your own launcher (by deriving from an existing one).

[minrk]

You can also change the launch command to a script to perform additional initialization, invoke profile/activate hooks:

#!/usr/bin/env bash -l
# put this in /path/to/my-launch-script
conda active some-env

export PATH="something:${PATH}"

# this always needs to be the end of the script:
exec jupyterhub-singleuser "$@"

# juptyerhub_config.py
c.Spawner.cmd = "/path/to/my-launch-script"

[willingc]

Closing this issue since it appears to be resolved. Thanks to all!