-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Proposal: introduce enhanced image resolution gateway API #2944
Comments
I don't have anything against option 1, except the field should be We can tackle the push part separately. I didn't really get Option3 as for anything more generic than |
Sounds good :) I think option 1 will be good enough to get at least the inspect functionality to go through buildkit. I think I made a typo above, we need to include the |
Stumbled upon this ticket when I was searching for something related to I think we also need to have a good look where we want this to live; there's also docker manifest, which has a large overlap with imagetools. Question is; are all of these commands to be considered related to build or also for other purposes? Basically; should this be something to be provided by the engine API or the buildkit API (if we want it to be handled daemon-side?) Or would there be ways to expose (these parts of) the BuildKit API through the JSON remote API? |
Hey folks, I think may have opened a ticket related to this, or at least further motivating the enhancements proposed here: @thaJeztah , in your search for docker and manifests, perhaps you could look over my ticket and suggest any workarounds?
@tonistiigi and @jedevc , I tried to developing a minimal go example to introspect what the Pseudo Minimal Example (not working)
package main
import (
"context"
"fmt"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
)
func main() {
ctx := context.Background()
// Create a new BuildKit client
bkClient, err := client.New(ctx, "tcp://0.0.0.0:1234")
if err != nil {
panic(err)
}
// Define the build
state := llb.Image("docker.io/library/alpine:latest")
def, err := state.Marshal(ctx)
if err != nil {
panic(err)
}
// Solve the build
_, err = bkClient.Solve(ctx, def, client.SolveOpt{}, nil)
if err != nil {
panic(err)
}
fmt.Println("Image built successfully")
// Use the ResolveImageConfig API
_, _, img, err := bkClient.ResolveImageConfig(ctx, "docker.io/library/alpine:latest")
if err != nil {
panic(err)
}
fmt.Printf("Image Config: %+v\n", img)
} |
This proposal suggests adding new/modifying existing APIs to extend the image resolution API.
Motivation
At the moment, to resolve images, the
ResolveImageConfig
API can be used - when passed an imageref
andplatform
, the API returns the image digest of the manifest, and the contents of the config object. However, this is not sufficient for all use cases: scenarios that involve listing all the available platforms by traversing the image index, or viewing layer information, or annotations attached at different levels than the config not currently possible.Additionally, because of this limitation, buildx must resolve images using methods imported from buildkit, instead of the API, resulting in the odd scenario where an image can be built+pushed using buildkit using registry information from
buildkit.toml
, but not inspected using the imagetools API.Option 1
Option 1 is the change of least resistance. We simply add new fields to
ResolveImageConfigResponse
that contains the index and manifest content:When querying an image config, we additionally attach the image manifest and index - this should add little overhead to the pull process, since we need to traverse to reach the image config anyways, however, depending on the size of the manifest and total number of manifests, each request may attach a lot of information that is never actually used.
Option 2
Option 2 requires some API modification and deprecation.
We rework the
ResolveImageConfigRequest
andResolveImageConfigResponse
to:This maintains a single endpoint with API compatibility with previous versions, but allows querying for not just the config, but the manifest or the image index. Additionally, this could easily be extended in the future with more precise queries for fetching annotations/attestations, etc.
While this maintains API compatibility, this requires replumbing large portions of existing go code, which is made harder by the necessity of returning the manifest digest, which only makes sense in the context of returning the config (since in the other cases, it is present either in the request, or the data response).
See a prototype here.
Option 3
Option 3 introduces a new set of APIs
Resolve
,Fetch
andPush
(for completeness), inspired by the containerd API.This is a generic fetch/push/resolve api, allowing querying and pushing from the client side. Additionally, this helps solve the additional buildx imagetools issue where manifests are created, and are pushed directly from the client, instead of through the server.
This places the traversal logic onto the client, and would allow implementing the
Resolver
API in the client, essentially allowing using buildkit as a pull-through tool.Conclusion
My personal preference is to implement option 3, and add the new APIs. This allows for far more flexibility than the other options in the future, and allows working from a clean design, that doesn't break any existing gRPC and Golang API contracts. Additionally, in the long term, the
ResolveImageConfig
API could be deprecated and reworked to be a shim layer over the new APIs.Comments appreciated, sorry for the long explanation, the options mostly walk through my thought process in thinking about solving this problem 🎉
The text was updated successfully, but these errors were encountered: