git url: support query form + support specifying checksum in query

[AkihiroSuda]

Split from:

• git url: support CSV form (also support verifying a tag with a commit hash) #5903

---

e.g.,

https://github.com/moby/example.git?ref=v1.0.0&commit=deadbeef&subdir=/subdir

# alias of ref=refs/tags/v1.0.0
https://github.com/moby/example.git?tag=v1.0.0

# alias of ref=refs/heads/v1.0
https://github.com/moby/example.git?branch=v1.0

Fix #4905, but the syntax differs from the original proposal.

The document will be added to
https://github.com/docker/docs/blob/main/content/manuals/build/concepts/context.md#url-fragments

[tonistiigi]

@AkihiroSuda Needs rebase after #5975

[tonistiigi]

One thing that is weird atm is the URL parsing in LLB (client lib and identifier) vs Dockerfile. The query form should really be only for Dockerfile/dockerui. In LLB the parameters are sent with attribute values instead. For historical reasons, the current LLB is kind of a mix where #ref is still part of the URL from initial implementation, but for example protocol of the URL is always git:// and the real protocol is sent as an attribute.

Currently, it looks like in LLB client we parse a full URL (with querystring etc.) but then we drop everything but Host/Path and send the original URL as attr (that I think will container querystring). Also #ref is not even part of the URL the user passes in the client function, but is glued to the identifier internally.

Then on LLB receiver side, this now handles querystring in this PR in the URL in NewGitIdentifier (although it looks that it would require client side to error for the querystring to be passed at all there), but actually the checksum gets set later from pb.AttrGitChecksum.

In order to fix this, I think we should separate these concepts. Assuming that we do need some level of parser in the LLB side, I think having two separate packages might be too much so maybe just two parser functions where the Dockerfile one would parse the querystring and other Dockerfile specific things and then internally call the more basic parser. Looking at the current return type, there is already some weird things there, eg. IndistinguishableFromLocal seems to be only used in Dockerfile, and UnencryptedTCP doesn't seem to be used anywhere at all.

I'm also ok with changing the signature for llb.Git, eg. adding GitSubdir() (maybe even GitRef()) so it is the same as other git properties instead of requiring microformat and URL parsing as input that gets converted anyway. On the identifier parser side, we do need to keep backwards compatibility though (but could deprecate the old behavior).

[tonistiigi]

This is the place where it is getting checksum from querystring in LLB while it really comes from pb.AttrGitChecksum

[tonistiigi]

@AkihiroSuda Any update?

[AkihiroSuda]

Rebased, but refactoring the parser may take some time. Unlikely in this week.

[AkihiroSuda]

Split the commit git url: rename GitURLFragment to GitURLOpts to #6012 for ease of merging.

Is the commit git url: support query form safe to cherry-pick too, ahead of the refactoring of the parser?