Nested LLB invocation support

[tonistiigi]

Adds a buildOp for nested builder invocation. Currently only nested llb is supported. In the end, this feature should be used with frontends.

Example:

go run ./examples/nested-llb/main.go | buildctl  build

This builds buildctl binary for all the currently open PRs for buildkit. The main.go script itself doesn't access github for getting the list of current PRs, that is done in a build step as well.

This is WIP. Some things are not working yet like detecting duplicate vertex statuses for progress. I added the required llb pkg methods with quick copy/paste from exec. I think this package needs some refactoring as ideally something like a nested build step would not be defined there at all, instead the package should be extendable so custom implementations with own marshal functions can be added. Done

[AkihiroSuda]

Interesting feature, do you have more usecases?

[tonistiigi]

@AkihiroSuda The usecases are:

• code reuse without copy/paste. Common cases can be written once, shared publicly and just referred inside build definition
• subcomponents/subprojects can define how they are built. For example if a dependency has a Dockerfile we can build it from that directly.
• detecting and generating build definition from the source

The current example is just something that can be easily verified and didn't require any new tooling. I don't think anyone would actually build go from source to generate the definitions.

[tonistiigi]

@AkihiroSuda I've fixed the issues and removed the WIP label

[AkihiroSuda]

We need consideration for #81 but lets merge this PR first.
LGTM.