JupyterHub authenticates login but doesn't redirect to user home

[patriciaferreiro]

Steps to reproduce issue:

1. Follow the Kubeflow Setup tutorial and apply default configuration.
2. Expose tf-hub-lb service as NodePort: ks param set kubeflow-core jupyterHubServiceType NodePort
3. Jupyter Hub login screen prompts but after authentication it just reloads the blank login screen.

Jupyter Hub instance Pod logs:

kubectl logs tf-hub-0 -n kubeflow

[I 2018-03-15 10:06:08.848 JupyterHub app:871] Writing cookie_secret to /jupyterhub_cookie_secret
[I 2018-03-15 10:06:08.894 alembic.runtime.migration migration:117] Context impl SQLiteImpl.
[I 2018-03-15 10:06:08.896 alembic.runtime.migration migration:122] Will assume non-transactional DDL.
[I 2018-03-15 10:06:08.919 alembic.runtime.migration migration:327] Running stamp_revision  -> 3ec6993fe20c
[W 2018-03-15 10:06:09.084 JupyterHub app:955] No admin users, admin interface will be unavailable.
[W 2018-03-15 10:06:09.084 JupyterHub app:956] Add any administrative users to `c.Authenticator.admin_users` in config.
[I 2018-03-15 10:06:09.085 JupyterHub app:983] Not using whitelist. Any authenticated user will be allowed.
[I 2018-03-15 10:06:09.142 JupyterHub app:1528] Hub API listening on http://0.0.0.0:8081/hub/
[W 2018-03-15 10:06:09.144 JupyterHub proxy:415] 
    Generating CONFIGPROXY_AUTH_TOKEN. Restarting the Hub will require restarting the proxy.
    Set CONFIGPROXY_AUTH_TOKEN env or JupyterHub.proxy_auth_token config to avoid this message.    
[W 2018-03-15 10:06:09.144 JupyterHub proxy:456] Running JupyterHub without SSL.  I hope there is SSL termination happening somewhere else...
[I 2018-03-15 10:06:09.144 JupyterHub proxy:458] Starting proxy @ http://0.0.0.0:8000/
10:06:09.385 - info: [ConfigProxy] Proxying http://0.0.0.0:8000 to (no default)
10:06:09.388 - info: [ConfigProxy] Proxy API at http://127.0.0.1:8001/api/routes
[W 2018-03-15 10:06:09.621 JupyterHub proxy:304] Adding missing default route
[I 2018-03-15 10:06:09.621 JupyterHub proxy:370] Adding default route for Hub: / => http://tf-hub-0:8081
[I 2018-03-15 10:06:09.638 JupyterHub app:1581] JupyterHub is now running at http://0.0.0.0:8000/
[I 2018-03-15 10:12:02.670 JupyterHub log:122] 302 GET / → /hub (@a.b.c.d) 1.96ms
[I 2018-03-15 10:12:02.722 JupyterHub log:122] 302 GET /hub → /hub/login (@a.b.c.d) 1.34ms
[I 2018-03-15 10:12:02.816 JupyterHub log:122] 200 GET /hub/login (@a.b.c.d) 45.07ms
[I 2018-03-15 10:12:09.971 JupyterHub base:346] User logged in: jovyan
[W 2018-03-15 10:12:09.994 JupyterHub configurable:168] Config option `singleuser_image_spec` not recognized by `KubeFormSpawner`.  Did you mean one of: `singleuser_image_pull_policy, singleuser_image_pull_secrets, singleuser_node_selector`?
[I 2018-03-15 10:12:10.017 JupyterHub reflector:98] watching for pods with label selector heritage=jupyterhub,component=singleuser-server in namespace kubeflow
[I 2018-03-15 10:12:10.019 JupyterHub log:122] 302 POST /hub/login?next= → /hub/ (@a.b.c.d) 78.56ms
[I 2018-03-15 10:12:10.076 JupyterHub log:122] 302 GET /hub/ → /hub/login (@a.b.c.d) 0.85ms 

Jupyter Hub services:

kubectl get svc -n kubeflow | grep hub

tf-hub-0           ClusterIP   None             <none>        8000/TCP       55m
tf-hub-lb          NodePort    10.108.109.231   <none>        80:32165/TCP   55m`

I figured the Config option `singleuser_image_spec` not recognized by `KubeFormSpawner` may indicate where the issue's coming from, already checked ks show default -c kubeflow-core output (attached) but wasn't able to spot any errors. Should I perhaps have modified any configuration parameters because of exposing the service as NodePort? Thanks in advance.
kubeflow_logs.txt

[pdmack]

Did you mean ks param set kubeflow-core jupyterHubServiceType NodePort? The command you gave should have been caught as an error by ksonnet.

ERROR Could not find component identifier 'tf-hub-lb'

Perhaps check your version of ks (mine is v0.8.0) and regenerate the app per the instructions. At any rate, setting the param on kubeflow-core and applying that component results in a working NodePort for the hub load balancer and spawned a server for me. HIH

[patriciaferreiro]

Absolutely, my bad. I'll edit my post with the correct command.

My ksonnet version is 0.9.1, I will perform a downgrade to 0.8.0 and update the issue. Thanks!

[pdmack]

Is it possible your browser is blocking cookies? JH needs to set session cookies, and I can replicate what you're describing by enabling that security measure. The dummy login just reloads the login page with empty fields.

[patriciaferreiro]

I am now using ksonnet version 0.8.0 but the problem persists. I have noticed that, after a successful login, Jupyter Hub doesn't redirect to the user's home:

[I 2018-03-15 10:12:10.019 JupyterHub log:122] 302 POST /hub/login?next= → /hub/ (@a.b.c.d) 78.56ms
[I 2018-03-15 10:12:10.076 JupyterHub log:122] 302 GET /hub/ → /hub/login (@a.b.c.d) 0.85ms 

Instead, it reloads the login screen.
It seems the problem lies in the CHP configuration, I'll try to troubleshoot some more and post results. Has anyone experienced similar issues?

[patriciaferreiro]

Checked Chrome's privacy settings and I already allow cookies (also third-party). However, the proxy works correcly on Firefox, so it may have something to do with its configuration. On a side note, I also deployed Jupyter Hub via Helm following the 'Zero to JupyterHub' tutorial some weeks ago and experienced no problems accessing the UI on both browsers.

[pdmack]

BTW, going back to ks 0.9.1 should be fine. Works for me.

[jlewi]

What is the CHP configuration?

[jlewi]

@patricia92fa What is the status of this? Were you able to get it fixed or work around it?

[DjangoPeng]

@patricia92fa Are you still blocked by this issue?

• If so, what's the status now?
• If not, please attach your solution to help others who face with the same trouble.