Test that uploaded crates can be parsed by all supported Cargo versions

[cuviper]

As noted in this urlo thread, Cargo <= 0.8 can't parse dependencies with pre-release version requirements, e.g. serde 0.9.0 requires serde_derive = "0.9.0-rc4", and this goes so far as to prevent such crates from being used at all, not even its older versions, not even if a new version is published and the one with the pre-release requirement is yanked.

I don't know if there's anything that can be done for that case at this point, but it would be nice to prevent similar accidents from happening in the future. AIUI, it's generally the intent that the Cargo.toml format should remain compatible with old Cargo, at least for parsing, even if that old Cargo wouldn't understand some of the fields. So I think it would be a good idea for crates.io to do a sanity check on uploaded crates, perhaps something like cargo generate-lockfile using whatever baseline cargo version we still want to support in the ecosystem.

(It's understood that not all crates can build/test with older rustc/cargo, especially given the existence of crates using nightly-only features.)

[alexcrichton]

I'm sort of surprised that this is so much of an issue, this basically means that we can't fix bugs in Cargo?

[cuviper]

That seems a rather extreme slippery-slope conclusion.

Taking a step back, to what extent does crates.io expect to allow usage by older Cargo?

[alexcrichton]

I guess put another way, let's say we want to close this issue as solved. Older cargo had a bug where it can't parse 0.9.0-rc4. Does that mean that we can never fix that bug? Should we forever be limited by what older Cargo does?

I'm all for supporting as many versions of Cargo as possible, but I don't think it should hold us back. Upgrading cargo isn't suppose to be hard, just a rustup update away.

[cuviper]

I don't have a problem with having fixed that bug in Cargo. The problem I see is that crates.io started publishing data that older Cargo can't handle, and we didn't realize it. And in this case, the effect is so nasty that even older crates that used to work fine are now made inaccessible just by having a newer version with this incompatible syntax.

So at a minimum, I'm asking for testing so we know before something breaks. From there, we can make deliberate choices about it. Maybe we raise the minimum bar for Cargo versions, but do so knowingly. Maybe it doesn't have to break at all, but some compatibility path can be added, like moving the new functionality to a new API version. It all depends on the nature of the change.

Upgrading cargo isn't suppose to be hard, just a rustup update away.

This is only true in the Wild West of a pre-distro Rust ecosystem. The continued success of Rust also means rustup won't be the only way to get rustc+cargo, and LTS distros are unlikely to keep up with every release. If a breaking change is going to happen, I'd rather see it made carefully and deliberately.

[TechnoMancer]

There was also the addition of quoted keys to TOML recently which is used for some cargo features. This however only prevents use of versions of crates that use them with older cargo rather than breaking the crate regardless of the version used like the issue that prompted this.

This would however be detected by automatic testing of crates' Cargo.toml, and I think doing that would produce a large number of failures for use of new cargo features.

[carols10cents]

I see a few issues here:

1. Is there something we can change about the crates.io API to enable old versions of cargo to install versions of crates with normal version numbers, even if those crates have other versions with -alpha, etc that aren't compatible, so that older versions of cargo can install serde, etc?
2. Can we set up infrastructure to run a set of operations (cargo build, cargo publish, what else?) using older cargo versions and the latest cargo version, with a variety of crate data, to catch when crates.io makes a change to its API that would be incompatible with an older version of cargo?
3. Bigger issue: Cargo is still, officially, "nightly"/unstable even though versions of it are installed with stable Rust. Furthermore, while LTS versions of Rust (and by extension cargo) have been mentioned from time to time, afaik there is no RFC proposing that we designate a particular version as LTS ever. While the ship has sailed on the versions of Rust/cargo included with Ubuntu LTS, could we, in the future, designate our own LTS versions for distros (and whoever else) so that everyone is clear about expectations, and so that we don't have to test against every version of cargo that has been released with every version of stable rust?

I think there should be another crates.io issue for 1, this issue should be for 2, and someone should start an RFC for 3 (i'm not necessarily volunteering, but i'm not NOT volunteering either)

What do yinz think?

[cuviper]

@TechnoMancer hmm, that's a good argument that I'm not quite targeting this at the right place. It would be somewhat OK, though not desirable, if some new syntax of Cargo.toml can't be handled by old cargo, as long as that old cargo can continue using what it was before. i.e. preserve access to old crates even if some new crates become inaccessible. But I want to make sure that old cargo can still deal with crates.io itself, which I guess is a question of the web API and the registry JSON format.

@carols10cents I think I agree with your breakdown, yes. In 2 I'd like to see compatibility testing of crates.io itself. Your cargo build and cargo publish examples are good tests of the consumer and producer sides respectively for when the API changes. But I think to catch a problem like 1, there needs to be live testing too, because it shows how something broke without any explicit format change. So when they went to publish serde 0.9.0, something would flag it to say "Hold on, if I put this in the registry JSON, old cargo fails -- red alert!" Or if someone's description is "Robert'); DROP TABLE Students;--", or who knows what the future holds...

3 is a bigger question, yes. However, I made a casual comment about this in the context of roadmap 1.0 crates, that Cargo isn't even there, and I was corrected that cargo the command-line tool is considered stable -- only the API as a crate is not stable yet.

[alexcrichton]

@carols10cents your strategy sounds good to me.

I'd also echo what @cuviper said in that while Cargo does not literally declare itself as 1.0.0 it's practically infeasible to introduce breaking changes to it now anyway.

[aturon]

@cuviper @carols10cents re: LTS, this is something the core team has long been interested in. If either of you are up for writing an RFC, I'd be glad to help mentor!