Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to using official kubernetes client library #43

Closed
yuvipanda opened this issue Apr 21, 2017 · 1 comment
Closed

Switch to using official kubernetes client library #43

yuvipanda opened this issue Apr 21, 2017 · 1 comment

Comments

@yuvipanda
Copy link
Collaborator

https://github.com/kubernetes-incubator/client-python didn't exist when we started this spawner. It exists now, and we should use it.

It has async support of sorts, but it merely just spawns a new thread per request. I'm not sure if this is a valid way to do this, given we make a ton of requests. We've already had problems with this (see #30) We have a few options:

  1. Determine that spawning a ton of threads is ok, and works fine. Test that we have connection re-use and things like that
  2. Implement the async-ness ourselves, with a threadpool or something like that
  3. Rewrite how we use the client - maintain a global, updated copy of all pods in namespace (with a list + watch), and use this copy to implement the polling operation. Do (1) or (2) for spawning and deletion.

I'd highly prefer (3) - that's what all the kubernetes components use, and it's also super efficient. It'll allow us to scale to a much higher number of users than we can today.

@yuvipanda
Copy link
Collaborator Author

Done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant