Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial cargo-plugin-fields RFC. #2776

Closed
wants to merge 3 commits into from

Conversation

@ratmice
Copy link

commented Oct 3, 2019

Rendered


First RFC, so undoubtably i've bungled the process at some point -- hopefully not too badly!
I did an initial prototype patches, and a feature request:

# Motivation
[motivation]: #motivation

Plugins sometimes need additional information beyond that specified in Cargo.toml. Existing information like package.metadata is relegated to a side table. The motivation for this equally, the ability to specify these plugin fields, and specify them _in place_.

This comment has been minimized.

Copy link
@chris-morgan

chris-morgan Oct 10, 2019

Member

TOML is flexible in how it lets you express things. All of the following are equivalent:

package.metadata.pluginname.key = "value"
[package]
metadata.pluginname.key = "value"
[package.metadata]
pluginname.key = "value"
[package.metadata.pluginname]
key = "value"
[package]
metadata.pluginname = { key = "value" }

I’m not feeling this motivation.

A more interesting concept (which would be a completely different RFC) would be putting this in dependencies.pluginname.metadata and directly exposing it to the plugin somehow, so that you could write something like this in Cargo.toml:

[dependencies.pluginname]
version = "1.2.3"
metadata.key = "value"

This comment has been minimized.

Copy link
@ratmice

ratmice Oct 10, 2019

Author

I think, I perhaps need to refocus this RFC to focus more upon using json-ld,
Toml is defined in such a way that every Toml document can be converted to a JSON document.

And refocus this on providing a translation layer from Cargo.toml to JSON-LD documents.
Where the keys them become "@pluginname:key", and the semantic differences between having an arbitrary values for fields.

For example in package.metadata, you can have

pluginname.foo = "0.0.1"

Where foo means nothing to pluginname, it expands it to package.metadata.pluginname = {name = "foo", version = "0.0.1"}

Ironically we can then give all cargo fields semantic meaning like "@Cargo:package" as a translation to [package] to highlight the semantic difference, the one key that cargo key I know of to which we cannot give semantic meaning to is it's [package.metadata] which is capable of being an arbitrary table. The only thing in a json-ld document we can treat it as is a json-literal. Where the field

"@pluginname:version" = "bar"

Would translate to json-ld triples encoding fields cargo doesn't have any prior knowledge of, but can be checked for validity none the less.

I'm not sure if it's best to recycle this based upon that, or start out with a new RFC proposal.

Edit: I would also add my overloading of "." in field name, where it doesn't expand to key-paths, and outside the usage outside is probably enough in itself to motivate the move to ":"


```Cargo.toml
[package]
"@EO13526.clearance" = "unclassified"

This comment has been minimized.

Copy link
@chris-morgan

chris-morgan Oct 10, 2019

Member

Why "@EO13526.clearance" rather than "@EO13526".clearance?


Removal of the warning is done recursively so
```
"@EO13526.clearance = { classification = "unclasified", history = [...]}

This comment has been minimized.

Copy link
@chris-morgan

chris-morgan Oct 10, 2019

Member

Missing quote in the key, though where I’m not sure.

# Future possibilities
[future-possibilities]: #future-possibilities
It would be really nice if in the future there was a `map` behavior

This comment has been minimized.

Copy link
@chris-morgan

chris-morgan Oct 10, 2019

Member

I can’t figure out from this description or your code what this is supposed to mean.

@Centril Centril added the T-cargo label Oct 10, 2019
@ratmice

This comment has been minimized.

Copy link
Author

commented Oct 12, 2019

I think it's probably best to close this for now, While it could be resurrected in the form of json-ld which explains the differences between the metadata keys, and the @pluginname prefixed keys.

A better way to achieve this explanation is to first implement the json-ld semantics for Cargo.toml, without adding any new features or user facing changes to cargo.

Once that is out of the way it'll be easier to see how these plugin fields differ semantically from the metadata ones. I don't believe trying to explain both at the same time, in the context of a new feature is a wise approach.

@psFried

This comment has been minimized.

Copy link

commented Oct 13, 2019

Thanks for writing this up! I think that extending Cargo and making it easy to incorporate rust into larger multi-language build systems are both important and interesting use cases, so I'm glad to see people thinking along these lines. I also think that these problems are really tricky, given the extremely broad range of use cases and stakeholders. All that is just to say that I would like to have a really clear understanding of the intended use cases for this type of proposal. Reading through this, I'm finding myself a little unclear about how this additional data might be consumed. Would it be expected that Cargo would provide this data to build scripts? Or is it intended to be indexed by crates.io in some way?

I think it would be really helpful if you could add descriptions of intended use cases. Even if this RFC doesn't end up being accepted, having a better understanding of what you're wanting to accomplish might also help inform future RFCs.

@ratmice

This comment has been minimized.

Copy link
Author

commented Oct 13, 2019

@psFried I'm not exactly sure how the tooling is going to end up at this point,
The bare minimum is that we'd add a new cargo command such as cargo json-ld
running cargo json-ld --dump would translate cargo.toml into a json-ld file,

  "@context": "https://crates.io/cargo/package.jsonld",
  "@id": "http://crates.io/crates/foo",
  "name": "foo",
  "version": "0.0.0",
}

That is currently the extent of my plans for it, The plugin fields would come with their own context,
And json-ld is well typed, so a consumer who understands the context, can check the validity of the (in this case) package for conformance to the type, and do something with it.

Additionally they can add their own contexts, so for instance:
cargo json-ld --dump | cargo json-ld --build could add their own context which contains information about the results of the compilation and spits out the original conversion Cargo.toml with additional stuff. Similarly cargo json --build-commands could augment it with the context of the rustc commands cargo build would invoke.

Exactly how this mechanism interfaces with the general cargo plugin mechanism, is kind of less important to me, than ensuring that under the hood it is all nicely typed, and plays well with others!

Beyond tools which consume and process the information as streams described above,
Each tool or context is responsible for defining it's own set of fields and meaning in the standard format. How they get consumed though is basically up to each tool capable of understanding the context.

The primary non-stream use case i'm after is being able to specify lints on dependencies and transitive dependencies, storing that information directly in Cargo.toml, and all the various checks that are run by the CI, so that CI and contributors can both run the same plugin/sets of commands locally and easily keep them in sync.

The exact mechanism for integration of TOML and how the mapping from fields to contexts at the moment is needed to be thought about. But a very important detail, is that cargo needs to ensure that the fields which it defines, conform to the context in the way it is defined.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.