Please provide way to parse -o=value as option taking value =value

[ijackson]

Hi. I saw this crate mentioned in a blog post and I like the idea. But there is a difficulty:

Argument parsers should be transparent as much as possible. Currently, becuase lexopt supports -o=value to mean "set to value", to unparse an unknown string (like a user-provided filename) it is necessary to always pass the =.

(The situation with short options is different to long options: supporting --long=value is completely unambiguous and simply a good idea.)

IMO the = is unnatural here. I'm not aware of many other programs which treat -o=value as setting the option to value. Almost all (including for example all the POSIX utilities) treat it as setting the option to =value. See eg the Utility Convension in the Single Unix Specification, or the manpage getopt(3)

And as I point out, handling = specially for short option values is not a cost-free feature (unlike with long options): it changes the meaning of existing programs. A shell script which takes some arguments and runs a lexopt-using Rust program, and passes filenames to it, must write the = or risk malfunctioning on filenames which start with=. Because the = is unconventional with a short option, the author of such a script probably won't have written it, so the script will probably have this bug.

And, within an existing Rust project, switching from another option parsing library to lexopt is hazardous because it will change the meaning of command lines which are accepted by both.

Could you please provide an option to allow lexopt to be used without this = on short option feature? I'm not sure if that would involve a ParserBuilder or whether you could just make it a configuration method on Parser.

Personally I think the default ought to be to not handle = specially in short options but that would be a breaking change.

Thanks for your attention.

[blyxxyz]

Hi,

Thank you for bringing this up! I've spent some time thinking about this issue but I don't think I've done a proper writeup.

I used to have the same opinion as you, which is why lexopt 0.1.0 didn't support -o=. It felt alien to me, and risky for exactly the reason you describe. I changed my mind due to #8.

clap does support this syntax (as does Python's argparse). Most Rust projects switching from another library will be switching from clap, so supporting it seems like the right default. (This was a concern for newsboat in particular.)

Furthermore, moving from a library that doesn't support the syntax to one that does is a small compatibility break (only weird values are affected), while going in the other direction is a big one (normal command lines are broken).

(This does create a ratchet where it's easy to switch to a parser that supports it and hard to switch away. That's a downside of the robustness principle.)

Argument parsers should be transparent as much as possible. Currently, becuase lexopt supports -o=value to mean "set to value", to unparse an unknown string (like a user-provided filename) it is necessary to always pass the =.

Note that even without that syntax the safest solution is to pass -o "$value". That works even if $value is empty, while -o"$value" misbehaves in that case.

Could you please provide an option to allow lexopt to be used without this = on short option feature?

I don't like doubling the number of potential cases, but maybe making it configurable is for the best.

A workaround for now is to use version 0.1.0.

I'm not sure if that would involve a ParserBuilder or whether you could just make it a configuration method on Parser.

A method on Parser should work for this, but that way it can be reconfigured in the middle of parsing, which could be a problem for any future configuration. I'll have to think about it.

[Mango0x45]

I would also really like to have a way to have -X=ABC be parsed as giving -X the value =ABC. Even if that kind of format is supported by other rust/python libraries, the fact of the matter is that it's different from the behavior of about 99% of command-line utility found on Linux/BSD systems, so all you're really doing is potentially confusing users of programs making use of this library.

[blyxxyz]

I'm waffling about whether to implement this. Are there known cases of this behavior causing problems for clap or argparse users? The closest I can find is fish-shell/fish-shell#8466 but that's not very strong.

[Mango0x45]

Any possibility of this becoming a thing? It feels really weird to not follow the standard getopt() behavior here…

[blyxxyz]

It does feel weird but feeling weird isn't enough. To justify it I need concrete problem cases, real-world software that didn't work right because of this syntax.

The syntax is supported by the most popular argument parsing libraries for Python (argparse) and for Rust (clap). Do you know if they have had feature requests for this, or if there are Python or Rust programs this has been problematic for?

[blyxxyz]

I found a concrete case! uutils has a flaky workaround for this problem in its implementation of cut (see uutils/coreutils#2424).

echo foo=bar | cut -d= -f2

This is a reasonable command that's used a lot, it's even in my shell history. And uutils is planning to move from clap to something lexopt-based. So it would be nice to support it.

For a single case some crazy try_raw_args workaround might be enough, and I'm unsure about the best API for this, but I'm going to give it more thought.

[Mango0x45]

I have recently also started using similar behavior with the column command to align assignments in vim:

Before:

int x = 5;
double y = 6;

After column -t -s= -o=:

int x    = 5;
double y = 6;

[ijackson]

It does feel weird but feeling weird isn't enough.

I provided a link to a formal specification.

The correct prior art to refer to is getopt_long, which established the relevant conventions decades ago. And of course, the standard behaviour of all the existing Unix utilities. (clap and Python are, sadly, not examples of best practice.)

[blyxxyz]

I'm aware of the prior art. lexopt has other deviations. POSIX wants you to put all options before all positional arguments. GNU lets you abbreviate long options. lexopt does not have those behaviors, unless you impose them manually.

(Existing Unix utilities are honestly a mixed bag, dd and find and ps and tar deviate heavily and many other utilities deviate more subtly. getopt was a late invention. The argument parsing code in V7 Unix is a mess, different for every utility.)

I don't want to comply with conventions just because they're there. I want to comply with conventions because it's helpful, because we're worse off otherwise.

I introduced this deviation in 0.2.0 because I knew for a fact that some people would be better off that way. So I asked to convince me that some people are worse off because of it. Like by finding someone on the Python issue tracker whose script didn't work because of this syntax. I know about the theoretical objections, so I looked for cases like that, but I didn't find them, so I wasn't convinced we were worse off.

I kept looking for over a year and yesterday I finally found a case in which this behavior made someone worse off. That's what I was looking for.