Trying to handle parse errors inside run (to use unknown options as free args)

[vindarel]

Hi,

I've been wanting to parse options in a lenient way, so that unknown options are kept as free args. I found about unknown-option and treat-as-argument, the only issue is that run, which calls parse-command-line, handles everything with a handler-case. So I can't put a handler-bind around run to do my logic.

This is what I want to do:

(handler-bind ((clingon.conditions:unknown-option
                   (lambda (c)
                     (format t "debug: ~a: treating it as a free argument.~&" c)
                     (clingon:treat-as-argument c))))
    (let ((app (top-level/command)))
      (clingon:run app)))

if you replace clingon:run by (clingon:parse-command-line (top-level/command) (list "whatever" "--xyz-unknown-option")) it works.

How could we allow run to be lenient?

option 1: wrap the parsing with handler-bind and add a key argument to the run function to tell how to manage the unknown-option, i.e. modularizing this:

(defmethod run ((top-level command) &optional arguments)
  (handler-case
      (handler-bind ((unknown-option
                       (lambda (c)
                         (treat-as-argument c))))
        (let* ((arguments (or arguments (argv)))
               (cmd (parse-command-line top-level arguments)))
             …

For completeness, we would need to do this for every condition, and change the method signature. Doesn't seem like ideal.

option 2: give the parse-command-line function as argument so I can provide my own.

option 3: I tried to locally override the parse function, with (flet ((clingon.command:parse-command-line (command arguments) …) (clingon:run)), to no avail, it's still using the original one. Maybe I'm doing it wrong.

option 4: copy-paste run and tweak it.

option 5 (brainstorming): somehow extract the parsing from run. We would have a chain of two functions:

(run (parse top/level))

so I could use my parse function:

(run (my-parse top/level))

---

More context (I'm brainstorming here too): my CLI looks like:

$ tool [options] free-arg

and this free arg names a lisp script that will in turn be loaded and executed.

I want to give more options to this script. I can use "--":

$ tool [options] free-arg -- -a

so far so good. But I managed to be able to call the script directly:

$ ./free-arg

it's still calling tool [options] free-arg under the hood though. This is where I'd like to not use "--", but give it options directly like a real script:

$ ./free-arg -a

if I do this, currently Clingon barks and exits because -a is an unknown option (if it was not defined with the options of tool).

There is the machinery to turn an unknown-option to a free-arg, but I don't see how to do it with run.

---

The tool is CIEL, I'm trying to ease CL's scripting facilities.

my 2c: https://github.com/vindarel/lisp-maintainers#dnaeon---marin-atanasov-nikolov

Thanks!

[dnaeon]

Hey @vindarel ,

Thanks for the detailed issue! :)

The existing CLINGON:RUN method is meant to provide error handling for the most common user-related errors, so that users of the clingon system only need to have (clingon:run top-level) instead of wiring all the error handling on their own. The current error handling done by the existing CLINGON:RUN method include:

• Not specified on the command-line required options
• Missing option values
• SIGINT (CTRL-C) handling
• Generic catch-all-errors handler, which prints the error and exits instead of dropping you into the debugger

That last point is the one that is causing you troubles, but it is there because I think it is a good default for most of the use-cases.

• https://github.com/dnaeon/clingon/blob/master/src/command.lisp#L778

In situations like yours I think it is best to have a custom CLINGON:RUN method where you can define what the behaviour of parsing and executing should be. That way you have full control over what is happening behind the scenes, but at the cost of having to have a sub-class of CLINGON:COMMAND with the new method behaviour.

If you don't want to go with a separate CLINGON:COMMAND sub-class and it's associated new methods here's a few things you can try out.

Option 1

The tool you have specified in the description looks like this.

$ tool [options] free-arg [additional-args]

Is tool here a clingon app, which needs to call an external tool free-arg with additional-args?

If that's the case you can have a command like this in your clingon app.

(defun exec/command ()
  "Returns the command for the `exec' command"
  (clingon:make-command
   :name "exec"
   :description "execute any command"
   :usage ""
   :handler (lambda (cmd)
              (let ((args (clingon:command-arguments cmd)))
                (format t "Executing command: ~A~%" (clingon:command-arguments cmd))
                (format t "Output: ~%~A~%" (uiop:run-program args :output '(:string :stripped t)))))))

You can copy-paste the above handler and register it into your existing clingon app and call it like this.

my-cli exec -- uname -a

This will still require that you use -- to provide all the arguments, though.

Option 2

Provide a custom behaviour for CLINGON:PARSE-COMMAND-LINE by using an AROUND method.

(defmethod clingon:parse-command-line :around ((command clingon:command) arguments)
  "Treats unknown options as free arguments"
  (handler-bind ((clingon:unknown-option
                   (lambda (c)
                     (clingon:treat-as-argument c))))
    (call-next-method)))

With only this change and the EXEC/COMMAND function from the previous option you should now be able to use the CLI tool without having to provide -- on the command-line.

Example usage of a CLI tool built with this code, where the exec sub-command treats all unknown options as free args.

$ cli-tool exec uname -a
Executing command: (uname -a)
Output: 
Linux <hostname> 6.2.11-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 13 Apr 2023 16:59:24 +0000 x86_64 GNU/Linux

Let me know what do you think about these two options.

P.S.: Thanks for listing me on https://github.com/vindarel/lisp-maintainers !

[vindarel]

Niiiice, your option 2 is clean and is perfectly what I was looking for. I had spotted the defmethod, but had not thought about an around method. Turns out you put a method for a reason :)

a sub-class of CLINGON:COMMAND with the new method behaviour.

good point too, though now I don't need it. It will help others to spot other ways of customizing Clingon's behaviour.

---

In your option 1:

Is tool here a clingon app, which needs to call an external tool free-arg with additional-args?

"tool" is the clingon app. "free-arg" is not an external tool, in my case it's another Lisp file, but we might see it as an external tool, because its set of options is not known in advance. My "tool" is a binary, its CLI usage is known in advance, and my goal (completed) was to call this other Lisp file with any options I wish. We have the luxury to avoid the double "--", that's pretty nice. I see a usability issue (the unknown argument appears before the "free-arg" script name) but I'll test like this.

Best,