Advanced routing context for integrations

[migmartri]

Currently, an artifact gets routed to a given attached integration if the artifact is from a known type (i.e CycloneDX SBOM), this rigid mechanism has some limitations.

Scenarios

Scenario a) Two SBOM materials defined in a contract are sent to the same dependency-track project,

So for example, if you are attesting two different SBOMs, they are both sent to the project defined during attachment. This means project override.

Ideally, we'd like these to be sent to two different projects.

Scenario b) Single SBOM material entry, but containing different SBOMs

Let's say for example that you have a single CI job that is syncing a set of upstream assets. This means that you are using the same contract but technically running it multiple times for each asset.

In the example below, we are running the same workflowRun 3 times, but in the context of a different asset. This run is attaching the sbom material and overriding the same dependency-track project, because of the same issue described above.

Ideally, we should be able to even though we are re-using the same material entry, attach it some context that then can be used for the final routing. In other words, choose the dependency-track projects based on the asset we are processing.

Solution

There are two three new implementation concepts relevant to this problem annotations, templating and filtering

Annotations

Annotations are a set of arbitrary key/values associated with a given material. These annotations will

• Be defined in the contract
• Might have a default value
• Could be set during attestation, i.e attestation add --name ... --value ... --annotations asset=nginx

Templating

Currently, when you attach an integration, i.e dependency-track, you can choose the a project name that will be automatically created, for example

$ chainloop integration attached add ... --opt projectName=bitnami-asset 

We could leverage the new annotations to in practice expand a dynamic project name, for example

$ chainloop integration attached add ... --opt projectName="bitnami-asset-{{.Annotations.Asset}}"

This will make the plugin to template the project name based on an annotation value resulting on pushing to bitnami-asset-nginx, bitnami-asset-apache and so on.

Filtering

With annotations + templating we could solve the scenarios described above, and in fact it's very useful when you have dynamic capabilities in the plugin and an unknown number of inputs.

But there is another feature that builds on top of annotations that could be used to choose what kind of artifacts can be sent to a given attached integration and these are filters.

Filters work similarly to kubernetes labels, insofar during attachment you could pick just the materials that match the criteria.

So let's talk for example the first scenario, where we have multiple SBOMs as part of the same attestation that we want to send to different projects.

In this case we could

• 1. annotate the SBOM materials one as controlplane and another as cas
• 2. Attach the integration multiple times with different filtering values, for example

chainloop integration attached add ... --opt projectName=controlplane -l component=controlplane
chainloop integration attached add ... --opt projectName=cas -l component=cas

Now, each attachment will be executed only if the annotations on the received materials make sense.

This feature is also useful to forward generic artifact types to given integrations. @gr0 you might be interested in this.

### Tasks
- [x] Implement material annotations (both static and dynamic)
- [x] Implement templating in Dependency-track
- [x] Document annotations, add contract examples and dep-track example https://github.com/chainloop-dev/docs/pull/144
- [x] Make sure that Chainloop [release process](https://github.com/chainloop-dev/chainloop/blob/main/.github/workflows/release.yaml) uses this feature (scenario a) https://github.com/chainloop-dev/chainloop/pull/283
- [x] Global annotations support https://github.com/chainloop-dev/chainloop/pull/284
- [x] Global annotations support in dependency-track
- [x] Document global annotations
- [ ] Expose annotations in integrations https://github.com/chainloop-dev/chainloop/pull/289

[danlishka]

Thank you for documenting these requirements here, Miguel!

As you know, I am developing a system that automatically synchronizes and attests a curated catalog of containerized applications, their corresponding SBOMs, and other evidence materials. I want to use one workflow to run my sync job periodically. Yes, this is a scenario B).

Annotations with templating will definitely help, but we may still need to include root-level annotations to define attestations properly. To clarify, I must include the materials id, name, and version in my sync contracts today to show them later in all notifications at the top or to properly integrate with the Dependency Track instance. By utilizing root-level annotations, we can effectively describe attestations and communicate through integrations.

Regarding filtering, I suggest the following additions:

• Org-level integrations. I can attach integrations once across many workflows and parametrize them with annotations and filters.
• Throttling integrations through filtering. I want to throttle some integrations. For example, I want to send notifications only for major releases and skip them for minor and patch versions.

Hope it helps.

[migmartri]

and other evidence materials. I want to use one workflow to run my sync job periodically. Yes, this is a scenario B).

Thanks for confirming!

Annotations with templating will definitely help, but we may still need to include root-level annotations to define attestations properly. To clarify, I must include the materials id, name, and version in my sync contracts today to show them later in all notifications at the top or to properly integrate with the Dependency Track instance. By utilizing root-level annotations, we can effectively describe attestations and communicate through integrations.

That makes sense. Definitely using annotations to add context to an attestations seems the right way.

Throttling integrations through filtering. I want to throttle some integrations. For example, I want to send notifications only for major releases and skip them for minor and patch versions.

This is interesting, this technically could be done by attaching an integration with a filter that checks for example a label called upgrade-type is major

Org-level integrations. I can attach integrations once across many workflows and parametrize them with annotations and filters.

I agree, in my head I've had some auto-attach rules that could be defined