Remove "Docker Content Trust"

[tianon]

As an opener, I want to make it clear that this is 100% a self-motivated change; I am not making this change on behalf of Docker Inc, nor does my opinion here represent Docker Inc in any official capacity (ie, if you're writing a splashy news article about this, you're barking up the wrong tree by attributing it officially to Docker Inc; I'm "rogue" here / doing this on my own time as a personally interested party).

My biggest motivation for making this proposal is frankly the state of the upstream Notary (v1) project. It has been completely unmaintained for at least a full year, and mostly unmaintained for quite a few years (ref https://github.com/notaryproject/notary/pulls?q=is%3Apr + notaryproject/.github#70). No matter what value this feature might have once had, it currently is vastly overshadowed by mass bitrot, and it is my argument/opinion that we are actively doing the community a very large disservice (and even harm) by continuing to "support" the feature in the Docker CLI.

Given the state of the upstream project, it is my belief that this should qualify for an exception to our regular "deprecation" process such that we remove the feature immediately, and IMO we could very reasonably even consider backporting this deprecation to any past supported branches.

Arguably, we should have some official means of integrating other "trusted image" solutions into the Docker platform, but IMO those belong in the Engine (unlike DCT which is entirely implemented in the CLI), and I see that (more complex) discussion as orthogonal to removing this bitrot.

There are still quite a few TODO items here (most notably that we probably need some period of time with no-op/warning/erroring --disable-content-trust=xxx flags and deprecation documentation). I'm also certain I missed a few things, as I was mostly doing a pretty serious hack job to see how difficult this would be, not focused on creating a 100% optimal change (and this touches so many parts of the codebase that it's frankly more than I'm comfortable determining by myself whether I've made the changes correctly anyways).

[tianon]

Heh, well, proof's in the pudding (I've lit CI up nice and red to prove that I've clearly missed some things).

I'm also definitely confused by go mod tidy with this change, because it wants to add github.com/docker/libtrust to the list of "required" modules after I remove all this, apparently thanks to github.com/docker/distribution/registry/client.test, which is certainly annoying, to say the least, but extremely odd at best.

[thaJeztah]

❤️ I think I had a branch similar like this at some point.

The "trust" code is definitely interweaved in many places; while no decision is made yet on the fate of DCT (still waiting on direction), I did start to somewhat untangle the trust code from "non-trust", with the potential to add a compile-time build-tag to disable it (replacing DCT code with stubs possibly). Some are merged, but I have some WIP changes stashed locally;

• cli/command: internalize and deprecate Cli.NotaryClient #5885
• cli/command: deprecate Cli.ManifestStore, Cli.RegistryClient #5889
• move some trust-related code to trust package #5894
• cli/command: remove NotaryClient from CLI interface #5876

As to libtrust (and friends); yeah, it's quite possible that notary implicitly bumped the version (or similar), and removing it now makes go mod re-resolve (minimum) versions; sometimes that can result in dependencies being "added", and sometimes dependencies are left behind, because with the (indirect) dependency that forced a higher minimum version "gone", go modules now considers our own go.mod to be the source of truth, which means that it considers dependencies listed to be a "manual bump" ("explicit") for an (in)direct dependency.

Sometimes removing those lines from go.mod (/ vendor.mod), then letting go modules re-resolve versions can help to make them dissolve, but sometimes that is troublesome, as it may now have to re-resolve using legacy dependencies it finds in the tree. Those dependencies may not have a go.mod, so now it will try "latest!", which can result in things failing (I recall notary -> github.com/docker/go-metrics -> prometheus -> otel, and github.com/docker/distribution -> github.com/docker/go-metrics -> prometheus -> otel (or grpc?)) being problematic paths in those.

[tianon]

the potential to add a compile-time build-tag to disable it (replacing DCT code with stubs possibly)

Can you elaborate on this? Why would we want to keep the code / add even more complexity here? (Whatever new things we implement, they're not very likely to match the usage patterns of DCT, nor do we want them to, because we'd want something that's actually enforced in the daemon, not just a CLI toggle like all this is, right?)