refinery_cli can not be built reliably or repeatably in ci over time.

[stevenj]

We recently had to make a patch to a product we released about 4 months ago. It used refinery_cli v0.8.14 in CI.
Even though its a containerized build and the version of everything is locked down, compiler version and the refinery_cli release version itself, it failed to build.

It built fine 4 months ago. There were no changes to CI in the mean time.

The problem here is caused by a number of factors.

1. The refinery_cli dependencies are really broad which means any and all updates to upstream libraries that occur at anytime mean we build different versions of v0.8.14 depending on what the transitive dependencies have done since the last time its built.
2. Breaking Change in cargo/rust speak does not mean "Your CI won't break with this minor point release" it only means the API surface is the same. Upstream dependencies can, and do, break CI if you lock a compiler version and not the dependencies.
3. There is no lock file on refinery_cli which means its not possible to cargo install --locked to get a repeatable build of the tool.
4. Since 0.8.14 there have been no binary releases of the refinery cli so we can't use binary releases.

Due to the recent npm supply chain hacks, we are being forced to ensure our CI pipelines are repeatable and all dependencies and tools are locked to specific versions. As it stands, i don't see a reasonable way to do this with refinery_cli The upshot is, if we get a security audit of our CI we will probably fail, because its possible for a compromised upstream dependency of refinery_cli to find its way into our pipeline at anytime, and that would be undetectable.

Ideally from an auditability and supply chain integrity perspective, there would be binary releases when a new version is tagged, and they could be independently validated against building refinery_cli itself because it included locked dependencies that match the binary release.

Admittedly there is a lot more that needs to be done to lock down our CI fully, but if the tool fundamentally can't be repeatably built then we have no foundation to build upon.

[jxs]

Hi, and thanks for the interest! Yeah we should have Cargo.lock file versioned, would that help your situation?

[stevenj]

I believe it would help yes.