Join GitHub today
Implement long running interactive session and sending build context incrementally #32677
This PR adds a new
Not ready for code review yet, but open for feedback on the design. Currently, this is missing cache handling features and validations for the fs change stream.
Client calls this endpoint and exposes features that daemon can call back to. The first use case for this, also included in this PR, is the implementation to send incremental context changes for the builder. There is work going on to use this for accessing pull credentials. Later it could be used for accessing secrets from the client, transferring build artifacts back to client etc.
When client calls
Client API package adds only one new method
The implementation for the incrementally sending filesystem changes is in
The feature is behind experimental flag so to test it, daemon needs to be started with
To invoke a builder with using the session stream instead of sending the tar context you have to set the
Currently, for testing individual components this will use the session but still transfer the files with a tar archive(so there shouldn't be meaningful performance gain). To use the protocol that can do
As discussed while working on credential pulls, the actual session ID should not be part of the docker remote context identifier: in the future we will want to leverage this session for other stuff than build context, and possibly we will have case where context streaming is off, but other features like credential pulls or copy-out would be required.
Additionaly, client-side services seem a bit clunky both to declare and to discover. I would really like an api on client like:
@simonferquel @dnephin @cpuguy83 @dmcgowan What's your thoughts on the transport interface for this? I only made the generic interface https://github.com/moby/moby/pull/32677/files#diff-8d234f28fa68cff9ee839315dbeaeb49R51 because I couldn't decide in favor of a single transport(grpc callbacks, websocket, single hijacked grpc stream, ssh etc). If we only support grpc then this could be described more with proto files. I still wouldn't prefer to use grpc generation directly because I think the callbacks should follow the
I also made another poc implementation using this, for exposing ssh signers to the builder with
I have added 2 PRs directly on Tonis branch to fix issues on Windows:
2 times, most recently
May 3, 2017
4 times, most recently
May 16, 2017
I've removed the WIP indicator, rebased after cli split and added the persistent cache and garbage collection. I removed the transport interface and replaced it with grpc only logic similar to what @simonferquel suggested. The discovery part is optional, independent layer. It is generated from proto but the grpc itself does not rely on it. gRPC itself only uses the generated proto code to connect a client with the handler.
I've separated out the implementation of client session itself and file syncing that is one example feature on top of that but they could be reviewed separately or the order which we use to merge actual features doesn't matter. I've also included the CLI update to ease testing, that I will remove before merge.
I am still not sure about the Supports method at the method level. If we want to keep it, we should at least encapsulate that in the fssession package (version negociation of client side services should happen at the protocol level). So from fscache we should just have a function call like:
This way client code does not have to know about the notion of method url etc.
In term of versioning, I propose that we experiment a little with the following scenario:
In the fs cache scenario, we have a quite easy scenario as both protocol provide the same value (with more or less efficiency), but if we take the exemple of authentication, we could have:
In the case where client cli only implements v1, a v2 daemon should implement v2 on top of a v1 client (getting oauth access token using AuthConfig provided by the v1 client), in a transparent manner