✨Support for SBOMs/Provenance Attestations

[sipsma]

Buildkit has support for attestations: https://github.com/moby/buildkit/blob/master/docs/attestations/README.md

But Dagger has not yet integrated that into our APIs.

Any implementation should integrate with environment Artifacts coming as part of Zenith.

[sipsma]

One user requesting feature here: https://discord.com/channels/707636530424053791/1138842816219459695/1138842816219459695

[jpadams]

Was mentioned by folks at KubeCon Chicago.

[jedevc]

Ok so, I had some ideas related to this 🎉 This is my proposal for what we should do with attestations in dagger - thoughts welcome ❤️

Note

I think we should pretty much follow the in-toto spec: https://github.com/in-toto/attestation/blob/main/spec/README.md.

Buildkit implements attestations that follow this pretty closely, however, today it only produces statements (that contain predicates). We'd also want to upstream/wait for signing to be done, so we could use the envelopes above as well.

I think this deserves some more careful planning, instead of just aiming for buildkit compatibility - buildkit just aims to produce artifacts, whereas dagger lets you define programmatic pipelines, so attestations should just be as manipulable as every other type in dagger.

So, I propose we add a new core type, an Attestation (exact details can be bikeshed):

type Attestation {
  """Various in-toto details"""
  predicateType!: String
  predicate!: String / File
  subjects!: [AttestationSubject]

  """Converts the attestation (maybe signing it along the way?) into a File"""
  asFile!: File
}

type AttestationSubject {
  """the name of the subject (e.g. image ref, filename)"""
  name!: String
  """a list of digests that hash the subject"""
  digests!: [String]
}

(Maybe we might not want subjects directly in the attestation here. I can see both sides - should the user who exports the attestation get to set the subjects, or should the attestation producer get to set the subjects?)

This is a thin wrapper over buildkit's result.Attestation (https://github.com/moby/buildkit/blob/7eb2c8e0676d0dadfa6cb8ee89ef7772958fb0d1/solver/result/attestation.go#L19-L29). Note that this Attestation type is an in-toto Statement - it's not complete until it's exported (at which point it could be signed, and wrapped in an Envelope).

Creating attestations

To create an attestation, I think there will be two main ways to do this.

The first way, is we'll support getting a Provenance attestation for every dagger object. Adding this to the graphql for everything feels a bit fiddly, so maybe the right way might be to instead have a global dag.Provenance(value any) that would manage this. This means that objects will need to track their origins - for some things this is trivial, like for Container, it's pretty much just the buildkit ref we keep (as well as all it's dependencies), for other things (like user module objects) it's a bit trickier.

This should produce an attestation object like above, that tracks all of the inputs, what operations happened, cpu usage, etc. Anything that buildkit tracks, we can track.

---

However, we shouldn't expect that our code generates all of the attestations (note that we have to generate provenance in the core API, since it's very very closely integrated with buildkit).

The way to allow user-definition of attestations, make much more sense as soon as we include zenith modules (we already have a trivy module for example: https://daggerverse.dev/mod/github.com/jpadams/daggerverse/trivy). Instead of just returning a string with the SBOM attestation, the module should be able to return our custom Attestation type!

Consuming attestations

Why go to all this trouble of a custom type? Because, of what we can do with them - each attestation should be tied to some output of the dagger engine (either Export/Publish/etc).

Attestations have an asFile method, which can be used to convert it to the core dagger File type, so it can be read/exported/uploaded/etc - just as any other dagger file.

Imagine the following go pipeline:

builder := client.Container().
        From("golang:latest").
        WithDirectory("/src", project).
        WithWorkdir("/src").
        WithEnvVariable("CGO_ENABLED", "0").
        WithExec([]string{"go", "build", "-o", "myapp"})

res, err := builder.Publish(ctx, "ttl.sh/dagger-test")
if err != nil {
    panic(err)
}

To handle the provenance, we could adapt Publish to return a rich object, so that res has Digest and Ref methods:

// get the provenance attestation for the builder
provenance := builder.Provenance().WithSubject(res.Ref, res.Digest)
_, err := provenance.Export("provenance.json")
if err != nil {
    panic(err)
}

// or even, maybe as a shorthand?
provenance := builder.Provenance().WithSubject(res)
_, err := provenance.Export("provenance.json")
if err != nil {
    panic(err)
}

Or allowing ourselves even more flexibility, we could even automatically construct an attestation for every Publish/Export (but it still makes sense to allow this manual construction, since users might upload to other things, like manually connecting to S3):

res, err := builder.Publish(ctx, "ttl.sh/dagger-test")
if err != nil {
    panic(err)
}
provenance := res.Provenance()  // this would already have res attached as a subject

An important note! This would create a separate attestation - it wouldn't be embedded in the result as buildkit does it. This would allow integrating with cosign attach nicely. Potentially, we could later add support for attaching inline signatures similar to how buildkit does, but even having worked on that upstream in buildkit, I'm skeptical we want to do that (at least in a first pass).

Implementation details

In terms of implementation details, the high-level is that first-off, we want to piggy-back on buildkit's work here as much as possible. Having worked on that upstream, I don't really want to re-implement the whole thing 😄 (it was an absolute pain to get right).

The problem with this is that we can't use buildkit out of the box; buildkit does all of the attestation-related magic in Solver.Solve (https://github.com/moby/buildkit/blob/7eb2c8e0676d0dadfa6cb8ee89ef7772958fb0d1/solver/llbsolver/solver.go#L523-L535). There are 3 main parts here:

• We record CPU/memory/etc usage where possible
• We merge all ResolveImageConfig and every provenance.Capture (from Sources) into a big provenance.Capture
• We then call the post-processors with the above data, which can then create the attestation and attach it to the Result (which triggers the exporters to attach this).\

(un)fortunately we never call this. Instead, we call manually create a solver bridge, and work through that. Because the code here relies on some buildkit private functions, we'll need to either: 1. use the solver directly instead of it's bridge (probably not what we want), or, upstream some changes so that we can call those private methods ourselves to construct the provenance.