Use serde for target spec json deserialize

[Noratrieb]

The previous manual parsing of serde_json::Value was a lot of complicated code and extremely error-prone. It was full of janky behavior like sometimes ignoring type errors, sometimes erroring for type errors, sometimes warning for type errors, and sometimes just ICEing for type errors (the icing on the top).

Additionally, many of the error messages about allowed values were out of date because they were in a completely different place than the FromStr impls. Overall, the system caused confusion for users.

I also found the old deserialization code annoying to read. Whenever a key! invocation was found, one had to first look for the right macro arm, and no go to definition could help.

This PR replaces all this manual parsing with a 2-step process involving serde.
First, the string is parsed into a TargetSpecJson struct. This struct is a 1:1 representation of the spec JSON. It already parses all the enums and is very simple to read and write.
Then, the fields from this struct are copied into the actual Target. The reason for this two-step process instead of just serializing into a Target is because of a few reasons

1. There are a few transformations performed between the two formats
2. The default logic is implemented this way. Otherwise all the default field values would have to be spelled out again, which is suboptimal. With this logic, they fall out naturally, because everything in the json struct is an Option.

Overall, the mapping is pretty simple, with the vast majority of fields just doing a 1:1 mapping that is captured by two macros. I have deliberately avoided making the macros generic to keep them simple.

All the FromStr impls now have the error message right inside them, which increases the chance of it being up to date. Some "from_str" impls were turned into proper FromStr impls to support this.

The new code is much less involved, delegating all the JSON parsing logic to serde, without any manual type matching.

This change introduces a few breaking changes for consumers. While it is possible to use this format on stable, it is very much subject to change, so breaking changes are expected. The hope is also that because of the way stricter behavior, breaking changes are easier to deal with, as they come with clearer error messages.

1. Invalid types now always error, everywhere. Previously, they would sometimes error, and sometimes just be ignored (which meant the users JSON was still broken, just silently!)

2. This now makes use of deny_unknown_fields instead of just warning on unused fields, which was done previously. Serde doesn't make it easy to get such warning behavior, which was the primary reason that this now changed. But I think error behavior is very reasonable too. If someone has random stale fields in their JSON, it is likely because these fields did something at some point but no longer do, and the user likely wants to be informed of this so they can figure out what to do.

   This is also relevant for the future. If we remove a field but someone has it set, it probably makes sense for them to take a look whether they need this and should look for alternatives, or whether they can just delete it. Overall, the JSON is made more explicit.

This is the only expected breakage, but there could also be small breakage from small mistakes. All targets roundtrip though, so it can't be anything too major.

fixes #144153

[rustbot]

r? @fee1-dead

rustbot has assigned @fee1-dead.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

This PR modifies run-make tests.

cc @jieyouxu

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

These commits modify compiler targets.
(See the Target Tier Policy.)

The list of allowed third-party dependencies may have been modified! You must ensure that any new dependencies have compatible licenses before merging.

cc @davidtwco, @wesleywiser

There are changes to the tidy tool.

cc @jieyouxu

[Noratrieb]

I am finding a lot of target json specs with values that no longer exist, even an entire test case based on the assumption that it exists while it doesn't. I am getting a lot more confidence in the error behavior thanks to this.

[aapoalas]

LGTM :)

[aapoalas]

thought: The error formats are pretty repetitive; I wonder if a few helper methods wouldn't perhaps be an improvement? Something like

#[inline(never)]
pub(crate) fn s_is_not_a_valid_value_for(s: &str, field: &str, variants: &[&str]) -> String {
  format!("'{s}' is not a valid value for {field}. Use {}", format_to_list(variants))
}

and then the same for the "use command to print allowed values" etc.

This'd probably also reduce the slight drift between the error messages; some say "for field. Use 'a', 'b', or 'c'." while others say `for field, expected 'a', 'b', or 'c'" or such.

[fee1-dead]

I'm thinking of maybe using a macro that both generates the match with its arms and also formats the error message.

[Noratrieb]

that'd be a neat improvement in the future, but I wouldn't want to refactor the enum parsing in this PR here

[fee1-dead]

I'm thinking of maybe using a macro that both generates the match with its arms and also formats the error message.

[Noratrieb]

@rustbot ready

[fee1-dead]

@bors r+

[bors]

📌 Commit dad96b1 has been approved by fee1-dead

It is now in the queue for this repository.