Proxy entries not cleaned up when single user servers are stopped

[yuvipanda]

I'm running the proxy and the hub separately in installs (this allows me to restart them independently). This leads to some issues when the hub is restarted but the proxy isn't.

1. Hub calls user.stop() on all currently running users
2. This stops all the running singleuser services, but doesn't actually remove entries from the proxy!
3. Hub comes back, correctly determins there are no singleuser servers running. But the Proxy still thinks they are, and so this state is mismatched
4. Next time when you try to spawn something, the proxy proxies it back to where the singleuser server used to run, and fails with a 5xx error.
5. Restarting the proxy clears the proxy route table, bringing it back in sync with what the hub sees. This 'fixes' the issue

This doesn't manifest itself in the traditional configuration of the hub managing the proxy, since the proxy does usually get restarted with the hub. It also doesn't manifest when c.JupyterHub.cleanup_servers = False, since single user servers aren't stopped when the hub restarts. So it only manifests when you are running the proxy and hub separately with cleanup_servers = True

I think the real fix is to also cleanup the routes in the proxy when cleaning up single user servers on restart.

[minrk]

Agreed. It checks that all the routes for running containers are present, and fills them in if not, but it doesn't do the opposite - finding routes for not running users, and removing those.