notebooks_dir ~ expands incorrectly

[parente]

How to reproduce the issue

Configure jupyterhub 0.7.0b1 with:

c.SudoSpawner.sudospawner_path = "/some/where/bin/sudospawner"
c.SudoSpawner.sudo_args = ['-nH']
c.Spawner.notebook_dir = '~/notebooks'

Try to login. Notebook server startup logs:

[C 2016-11-21 12:32:15.936 SingleUserNotebookApp application:91] No such notebook dir: '/home/pparente/~/notebooks'

What you expected to happen

Path should be expanded properly.

What actually happens

Path is expanded but also gets the ~ part tacked back on.

Share what version of JupyterHub you are using

0.7.0b1

I put a print in the jupyterhub-singleuser script and confirmed that it is receiving --notebook-dir="~/notebooks" as in sys.argv. So it appears the incorrect expansion is happening somewhere after that.

[willingc]

Note to self: I'm thinking this may have something to do with changes in #748

[parente]

One diff I noticed while debugging, the command line quoting has subtly changed:

0.6.1

Spawning /opt/conda/bin/sudospawner-singleuser --user=bob --port=48512 
--cookie-name=jupyter-hub-token-bob --base-url=/user/bob --hub-host= 
--hub-prefix=/hub/ --hub-api-url=http://127.0.0.1:8081/hub/api --ip=127.0.0.1 
'--notebook-dir=~/notebooks'

vs

0.7.0b1

Spawning /opt/conda/bin/sudospawner-singleuser '--user="bob"' 
'--cookie-name="jupyter-hub-token-bob"' '--base-url="/user/bob"' '--hub-host=""' 
'--hub-prefix="/hub/"' '--hub-api-url="http://127.0.0.1:8081/hub/api"' '--ip="127.0.0.1"' 
--port=37437 '--notebook-dir="~/notebooks"'

[parente]

Shell quoting changed in 7da7f7e

[parente]

If I change my notebook config for 0.6.1 to simulate the extra double quotes found in 0.7:

c.Spawner.notebook_dir = '"~/notebooks"'

then I can reproduce the bug with the 0.6 code base:

[C 2016-12-01 04:08:18.659 SingleUserNotebookApp application:91] No such notebook dir: '/home/bob/~/notebooks'

So I'm certain the new quoting is causing it.

I've traced the --notebook_dir="~/notebooks" value all the way through to jupyterhub-singleuser's main where I can see it's still not expanded to the incorrect value. jupyterhub.singleuser doesn't appear to have any code that touches the notebook dir, which suggests it's something in the notebook source code that's tripping up on expansion due to the extra quotes.

[parente]

https://github.com/jupyter/notebook/blob/e6fb7e65e94f6dd94fc5aba0cbe4bd3065e19623/notebook/notebookapp.py#L910

value = '~/notebooks'
os.path.abspath(value)
/home/bob/~/notebooks

Without the extra quotes, something (the shell probably) implicitly did ~ expansion in 0.6. But now the value reaches the abspath call here unexpanded, and winds up just getting prefixed with the cwd. In fact, I can hit this problem with notebook 4.2 with:

jupyter notebook --notebook-dir='"~/notebooks"'

[C 23:24:27.458 NotebookApp] Bad config encountered during initialization:
[C 23:24:27.458 NotebookApp] No such notebook dir: '/home/bob/~/notebooks'

Looks like we just need to do an explicit os.path.expanduser somewhere.

[minrk]

@parente I think you're exactly right that we need to do an expanduser somewhere. Putting it in the single-user server seems like the right place.

[minrk]

#890 calls expanduser in single-user server. Perhaps that should work.