Brainstorm: What would a lexopt based clap look like?

[epage]

Very cool, I like the API a lot. I keep wishing I had more time on my hands to make something analogous to this as an internal clap API that all the clap features are built on top of which would allow using cargo features to aggressively trim down clap's features in an opt in/out manner (and thus be able to get that sweet small binary size).

https://www.reddit.com/r/rust/comments/oley2c/lexopt_a_minimalist_pedantic_argument_parser/h5evqhk/

What would this look like for clap?

[epage]

I was giving this some thought in terms of the impedance mismatches between clap and clap_derive. For example in #1104, features to help the builder API would do nothing for the derive or #2255 where the surface area of clap is so large, it makes it hard not to panic in some case in the derive. In general, having several ways of generating an App and parsing make it hard to document clap.

This also leads to performance loss. clap_derive can't take advantage of everything being a &'static str,we have to both parse and validate (which effectively is parse) rather than doing them in one step, etc.

What if we had

• clap_lexer: a lexopt like subset of clap that just handles all of the weird cases for walking parameters, without any rules knowledge, etc. This can serve as a platform for crates creating their own way of doing arg parsing without reinventing the wheel.
• clap_core: contains
  • Introspection traits (AppView, ArgView, SubcommandView, etc) and needed types (e.g. AppSettings)
  • Parsing traits (Parser, Args, Subcommand)
  • A way to serialize these using serde
  • Error reporting
  • Help generation based on the introspection traits
• clap_derive
  • Derives AppView/ ArgView, etc for the trait
  • Derives a Parser trait uses clap_lexer to parse everything into a generated version of a struct where every field is Option. We then run validation rules on it to convert the Option version into the real version
    • Basically, serde would be the inspiration
    • The basic building block would be Action.
      • These would basically be what we run in the cargo_opt match-case for an argument
      • We'd infer the action from the type, provide a library of actions the user can specify, and allow the user to specify their own action.
• clap_generate: operates on the introspection traits
• clap
  • Like clap today, based on clap_core
  • The builders would also implement serde deserialization

For crates that wanted to support either clap or clap_derive, they could implement all of the relevant traits. Derive's flatten and subcommand would work off of the traits. claps builders would support generating args from the introspection traits and provide a way to generate data from an arg matches.

Transition plan:

1. Pull out clap_lexer
2. Define *View traits and move to them, internally, as we iterate on the interface
3. Implement a new derive crate (what to name it so we can do a gradual transition?) that uses these building blocks

• Make *View* traits public

4. Update clap_generate to the *View traits
5. Iterate on derive / builder interop

[pksunkara]

Definitely something that we want to tackle after 3.0. There are a few discussions around this in other issues IIRC.

[epage]

Yes, I agree with after 3.0, I just wanted to have a place to start collecting these thoughts.

[ducaale]

It might be good to involve the author of lexopt (@blyxxyz) in this discussion.

[epage]

While I agree, I doubt we'll be able to base clap directly on lexopt, at least initially. I assume we'll gradually refactor towards this state rather than developing a crate to meet all of clap's needs and have a flag-day switch over. It'll then be a question of how much lexopt wants to complicate itself to meet the needs of clap.

[blyxxyz]

Basing clap on lexopt doesn't seem reasonable. It makes some tradeoffs in favor of direct "manual" usage instead of indirect library usage (e.g. borrowing so you can match against string literals), and it deliberately doesn't support everything clap needs (e.g. variable number of option arguments, -s=value). And in the end lexopt is pretty small so there's not that much to gain.

I like the idea of clap_lexer, though, so I'll follow this discussion with interest.