Add ability for users to elide ':' or '=' when CLI authors pass a

[c-blake]

non-empty partial symbol table. Behavior should be identical to the
old behavior if empty partial symbol tables are passed. "Partialness"
of the symbol table refers to the fact that one need only specify
option keys that are toggles/booleans/do not take arguments, hence
the "NoArg" suffixes in shortNoArg and longNoArg.

commandLineParams() returns seq[TaintedString], so use that consistently
in getopt() and initOptParser(seq[TaintedString]) dropping the taint at
the quoting stage just as with the paramStr() logic.

Fix capitalization inconsistency of cmdLongOption.

Export OptParser.cmd and OptParser.pos so that, at least in principle,
users of this API can handle "--" option processing termination or some
"git-like" sub-command stop word with a separate option sub-syntax.
{ Eg., case p.key of "": echo "trailing non-option args: ", p.cmd[p.pos..^1]
or case p.kind of cmdArgument: if p.key == "mysubcmd": .... } Really,
searching for the last delimiter before p.pos is probably needed to frame
the trailing text..Not the nicest API, but still possible with effort.

This is a follow up to https://forum.nim-lang.org/t/3592 where it sounded
like it would be welcome.

[c-blake]

There are obviously some judgement call choices in this idea. E.g., one could flip the sense of the partial table to have the CLI author specify which options take args instead of which do not take args. In my experience, programs with many option keys (like, e.g., the nim compiler itself) are about 2/3 option keys with args. So NoArgs is the "smaller" case needing less maintenance. It would not be hard to take either kind of list -- taking and not taking args, or some other parameter determining the logical sense of the tables and so on and so on, but that can get complicated to explain and CLI authors can mispredict which fork is easier for them (e.g., starting off with mostly toggles and evolving to mostly argument-taking). Just fixing on the "special" case of no arguments seemed simple, effective, and maybe on average ~3X easier than the typical full symbol table.

[c-blake]

Also, I am happy to add some material to the documentation comment/changelog.md and assume you would want that, but I figured first I would see if you liked the approach.

[Araq]

this should be of type set[char]. longNoArg should still be a seq to avoid the dependency I guess. Also @[] is easier to use.

[Araq]

Same here, shortNoArg should be set[char].

[Araq]

In my experience, programs with many option keys (like, e.g., the nim compiler itself) are about 2/3 option keys with args

Yes, plus IMO keys with args are to be preferred (--switch:on|off) for flexibility.

[c-blake]

I made the requested changes. Personally, I find string constants easier, but I can see both sides.

This is slightly off-topic, but while I have your parseopt.nim attention, I was wondering what you think of the idea of having more principled/documented quoting by using os.parseCmdLine to tokenize any cmdline strings into cmdline seqs/iterators before parseopt picks them apart into option keys, vals, and regular arguments. On Windows this would seem cleaner/more complete/less fragile. On Unix the creating process already does that tokenization. If avoiding two passes over the command is desired os.parseCmdLine could be turned into an {iterator, proc} pair.

[Araq]

I was wondering what you think of the idea of having more principled/documented quoting by using os.parseCmdLine to tokenize any cmdline strings into cmdline seqs/iterators before parseopt picks them apart

Yeah, probably we should do something like this...

[Araq]

Personally, I find string constants easier, but I can see both sides.

Well ... if we're serious about Unicode set[char] cannot work but then you need to use an UTF-8 iterator over the string constant...

[dom96]

IMO we shouldn't overcomplicate parseopt like this. What happened to the stance of no symbol table in parseopt?

[c-blake]

Well, it's only a fully optional partial symbol table with the only extra work being for boolean/toggle/no-argument option keys. If CLI authors want the old behavior, the defaults give it. If CLI authors want to provide CLI users with more POSIX-style option syntax then this PR enables that, too. Do you really want to force CLI authors wanting to provide more standard/common option syntax to use external to libraries/roll their own?

[Araq]

What happened to the stance of no symbol table in parseopt?

The PR seems simple enough and I think parseopt is not flexible enough for a standard library without this feature. Yes, I changed my mind.

[c-blake]

Ok. Well, I documented the changes for this PR. This should be purely added functionality not a breaking change.

Doing the full tokenization first with os.parseCmdLine as mentioned above in this thread would basically be a re-write of the parser against input token streams vs current input string. While we both think that is probably a good idea, it is also a much less simple PR.

[c-blake]

Oh, one thing the documentation reminded me to ask you about is whether you like NoVal/NoValue better than the current NoArg as suffices for short & long. "Argument" is often overloaded/used a couple of ways in this context (non-option arguments and argumetns to option keys) which can be potentially confusing.

[c-blake]

I went ahead and optimistically changed to NoVal and consider this PR good to go unless you really loved the "NoArg" naming. Other possibilities you might like better might be "Bool" or "Toggle" - if you prefer to refer more to the semantics of the option keys than the syntax of their parsing. It is a syntax optionality adjustment, though. The naming doesn't matter much to me, but we probably should not change once it's settled.

[mjoud]

This is a nice improvement but would it be possible to add some additional usage examples?

[c-blake]

@mjoud - fair enough. The theme of the documentation for this module seems to be to keep things pretty terse. I added one for the final getopt iterator (and altered the existing getopt example to have 'l' be a value-free short option so they make for an easy side-by-side. Added a few more doc snippets near procs, too.

An example handling "--" would be pretty complex. I think it would be better long-term to just add such handling to the parser to switch over to a cmdArgument-only mode, but I have been trying to keep this PR small.

[ghost]

is provided command users need not maybe this can be changed?

[Araq]

This requires a test case, apart from that it's fine.

[c-blake]

Ok. Done.

It would be nice someday to re-do the parser to be token-oriented from os.parseCmdLine or os.commandParams rather than the current character-oriented way. I basically did that parser already in https://github.com/c-blake/cligen/blob/master/parseopt3.nim and am happy to donate it. It also emits better error messages for users mistyped commands. That requires more testing on Windows in RTL mode than is easy for me to do, though.

Also, this is way off-topic, but there is a comment in the code related to POSIX not making argv available that indicates you may be interested in a trick I figured out about 20 years ago. You can infer argv from the C global environ on virtually every platform (I tested like 7 or 8 OSes back in the late 90s when Unix diversity was higher). I don't know about Windows (where it isn't necessary), but all the various Unix kernels just lay out the command argument memory area the environment variable memory area. The only variation I saw was whether the array of pointers comes before or after the area of string data being pointed to (and only HP-UX varied from the pack in that way).

[Araq]

all the various Unix kernels just lay out the command argument memory area the environment variable memory area.

Ok, but I don't want to make usage of this knowledge. :-)

[c-blake]

Yah...I suspected you would say that. :-)