Should renovate consider trust?

[thepwagner]

Systems like Docker Content Trust and sigstore provide an additional means to verify artifacts were published by a trusted source, focused on the docker ecosystem. PGP adds an additional layer of trust to git (commit/tag signing), github-releases (e.g. tianon/gosu), or some future datasource that fetches releases from a http file listing e.g. (apache ant) / sample implementation from a Go codebase.

As a renovate user,
I want to configure [B42F6819007F00F88E364FD4036A9C25BF357DD4] as the trusted keys for tianon/gosu,
So that I only receive updates signed by the tianon.

If Renovate were to consider artifact trust, it could not propose upgrades to unsigned/untrusted versions.
One approach would be filtering untrusted artifacts in the Datasource abstraction. In another archived Go codebase, I found swapping querying a Docker registry with a Notary server to be a smooth transition. Cosign also seems capable of listing versions from the signature database, potentially avoiding the Docker registry.

Reasons not to: scope creep.

• Defining the mapping between dependencies and their trusted keys is complex. Fetching keys will be an additional dependency/source of error at runtime.
• Verifying signatures will slow Renovate down for users that opt in by providing keys. The implementation might involve forking to gpg or cosign binaries.
• Containers and compiled binaries lead the way. Verification would only be used when renovate is acting as "package manager", and may contradict signing features within package managers e.g. gradle.
• Concerned users could just do this in a CI job triggered by Renovate's PRs.

I suspect Renovate doesn't want this scope, but I didn't see any other discussions/issues so I raised this idealogical question: should Renovate include dependency provenance, or should that be left to the repo?

[rarkins]

Really interesting topic. I'm not an expert on signing in particular and don't have a strong opinion, but I'd say that in most cases I've learnt that visibility is important even if an upgrade is not acceptable for some reason. As in: it's usually useful to know that an upgrade exists which you're not taking, rather than it being simply invisible. And I'd say especially so if the release is rejected due to reasons which might be security-related.

So one simple question you touched on is: If Renovate was agnostic to signing and it required another tool to do a CI check of Renovate's PRs, then is there much downside to that?

One argument in favor of not doing it in Renovate: you probably want to be doing signing verifications in CI regardless of whether Renovate created the PR or a developer.

One reason why you might want Renovate to be signing-aware would be if e.g. you are on 1.0.0, and 1.1.0 is signed while 1.2.0 is incorrectly signed. If users think "I'd like Renovate to pick 1.1.0 and ignore 1.2.0" then that implies that Renovate needs to be checking newer versions for signing and then "working back" through the list. If instead you were already on 1.1.0 and 1.2.0 existed but was unsigned, what would users then want? For it to be ignored, or for it to be raised but with warnings?

[thepwagner]

I've learnt that visibility is important even if an upgrade is not acceptable for some reason

I agree - if signature handling does fit in Renovate, perhaps it belongs later in the process than the Datasource. The Manager's ArtifactError has a nice pattern to communicate "I tried, but verification failed", similar to lockfile failures.

That might also clarify the expectations in the case you mentioned: Renovate will follow existing versioning, but any PR proposing 1.2.0 will be flagged as probably not a great idea to merge as-is.

If Renovate was agnostic to signing and it required another tool to do a CI check of Renovate's PRs, then is there much downside to that?

From a holistic perspective, Renovate having a definition/interface for signing might drive alignment between those downstream tools. Projects should be releasing signed artifacts, and Renovate is a potential force to incentivize. Who wouldn't want a ✅ in a potential new "Signature Verified" column.

Customers of hosted Renovate might like the idea that potentially malicious releases are filtered from CI, which might have additional permissions or be in-house.

One argument in favor of not doing it in Renovate: you probably want to be doing signing verifications in CI regardless of whether Renovate created the PR or a developer.

Agreed here too! One shouldn't trust Renovate, but it would be interesting if Renovate could present signature verification results.
My use cases (container images and URL-backed binaries) mean that Renovate would also need to do the signature verification. I think input from a user using verification via package manager would be an interesting point of view… if they even exist. I'm assuming the current experience might also raise ArtifactError if a signature fails during lock file generation.

[rarkins]

I agree - if signature handling does fit in Renovate, perhaps it belongs later in the process than the Datasource. The Manager's ArtifactError has a nice pattern to communicate "I tried, but verification failed", similar to lockfile failures.

I like this idea better. e.g it could be an optional, additional status check.

From a holistic perspective, Renovate having a definition/interface for signing might drive alignment between those downstream tools. Projects should be releasing signed artifacts, and Renovate is a potential force to incentivize. Who wouldn't want a ✅ in a potential new "Signature Verified" column.

Good point, although this idea perhaps expands the scope a little. e.g. going from user-configure per-package "verify these signatures" to the idea that Renovate adds a check automatically, similar to GitHub marking commits as "Verified"?

Customers of hosted Renovate might like the idea that potentially malicious releases are filtered from CI, which might have additional permissions or be in-house.

FYI at WhiteSource we acquired and are now extending a security service called "Diffend". The idea of it is to flag suspicious packages in real-time and then manually review them as quickly as possible. Switching from signed to unsigned or verified to unverified would definitely be the type of signal which fits in there. One of the concepts of Diffend is that it integrates with the package managers itself to protect all systems (developer, CI, even Renovate) from downloading suspicious packages. It is therefore more wider protection than Renovate itself filtering/skipping.

One argument in favor of not doing it in Renovate: you probably want to be doing signing verifications in CI regardless of whether Renovate created the PR or a developer.

Agreed here too! One shouldn't trust Renovate, but it would be interesting if Renovate could present signature verification results.

Actually I wasn't meaning directly not to trust Renovate (but don't object to the point). I was meaning that Renovate might not be the only source of package additions or upgrades in a CI system and therefore you need all PRs reviewed/scanned and not only Renovate's. Maybe Renovate could be the one to do that, or maybe it's a separate concept.

[thepwagner]

user-configure per-package "verify these signatures" to the idea that Renovate adds a check automatically, similar to GitHub marking commits as "Verified"?

I was thinking as part of the current Package, Update, Change table that Renovate presents, e.g.:

To populate that column, Renovate would need to:

1. Integrate with signature support where it exists in current ecosystems (e.g. for bundler, maybe the upgrade is attempted with --trust-policy HighSecurity, then retried at MediumSecurity if a signature issue is raised. For Docker maybe it queries notary directly, or naively pulls with DOCKER_CONTENT_TRUST=1. For gradle... ).
2. In places where there is no package manager (e.g. regex), Renovate would need the "new" user-configured per-dependency keys. It's just awkward that this is the use case I want the most :)

I was meaning that Renovate might not be the only source of package additions or upgrades in a CI system and therefore you need all PRs reviewed/scanned and not only Renovate's

Sorry for the poor phrasing, that's exactly what I meant too! Renovate is an awesome tool, I trust it with my repo contents. "One shouldn't special case Renovate's PRs" is 💯 .

[rarkins]

I think what we're looking for is incremental improvement, by which I mean:

• Fail anything that gets worse (a package going from signed to unsigned)
• Ignore anything that remains neutral (e.g. unsigned -> unsigned)
• Applaud anything that gets better (e.g. unsigned -> signed)

I would prefer not to "fail" neutral ones because it might create too much noise that you miss the important ones which get worse.

BTW Renovate tends to not run commands like bundler install x and usually ends up doing:

• Manual edit of the manifest file
• Child process running of package manager such as npm install or bundle lock to update the accompanying lock file

^^ Point being, it may require a rearchitecture to e.g. use Bundler's out of the box trust policy capabilities.

Another approach: assuming most registries are immutable, that should mean that "was this release signed or not?" should be immutable too. In other words, we only need to calculate it once, and calculation should be independent of the repo which happens to be run by Renovate first.

We could probably validate signing once-per-release within datasources as long as we can cache it indefinitely (unlike release lists themselves, which we typically cache for 15-60 minutes). Related to that: #11232