Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Switch to using kubernetes_asyncio client #744
Currently we use the official kubernetes python API Client. This is a purely synchronous client, and we must use threadpools / threads to use it from our tornado app.
We should experiment with using https://github.com/tomplus/kubernetes_asyncio instead, which is the asynchronous version of the library. This should (hopefully) simplify our code and let us stop thinking about threads.
We should consider how well supported the async k8s library will be (in a year or two from now). Right now it seems like a hobby project with one main contributor.
In the past we have had some problems with leaks/things breaking because the official library had bugs/made changes which causes trouble "in real time", compared to getting thread pools right for which you have infinite amounts of time. I prefer that we struggle during coding to in real time. Especially as the thread pool pattern is well established and setup in our code base already.
Is there a particular bug that would be fixed by switching or feature it would enable?
Right now my assessment is:
Should we put this on our "to watch" list and assess how the project has progressed in attracting more maintainers in a few weeks or months?
referenced a pull request that will
Dec 2, 2018
I stepped back from this a little bit and realized what really needs to happen is a port of the 'SharedInformer' concept from the k8s go client to Python. It is an encapsulation of a list + watch that handles coalescing, retries and resyncs for you. The existence of this library is why most k8s related stuff is in Go, and why they function pretty well. We've fixed a number of issues in kubespawner where we (mostly @minrk) learnt why we need an encapsulation like this.
More information on the SharedInformer is at:
I don't believe it is possible for a human being to write a correct implementation of this while mixing threads & async.
re: compatibility, I don't think that's a problem. When running inside a k8s cluster, clients always use
Maintainership is the only long term issue, but I'm personally committed to contributing to this too if necessary. The following things make me not too concerned about it:
Kubespawner is a bit more complex than binderhub, so I wanted to try this out on binderhub to gain experience running it to see how it goes. BinderHub's use of the k8s API is fairly minimal, and I am 100% sure there are bugs here we aren't noticing.
Since kubernetes_asyncio wants to be backwards compatible with the synchronous library, the changes are somewhat minimal. I've started work on this with #745. I need to figure out all the threads in use and make sure I've converted them right, and understand enough of the testing infrastructure to fix that too.
In parallel, I'm also working on a separate library that should implement sharedinformers - https://github.com/yuvipanda/kubernetes-informers. This should have thorough tests for various issues that might arise. I'm happy to wait for that library to mature before using that in BinderHub if that sounds better.
@yuvipanda Thanks for nice introduction
Yes, my library is very similar to the official library which you're using. It's also generated from a specification but it uses the