All commands should go through same shlex parsing

[tleonhardt]

Currently arguments for commands using the options decorator are split using shlex.split(), however arguments for commands which do not use the options decorator are not split in this fashion.

Ideally, arguments for all commands would go through a more similar processing chain.

The real question, is where is the best place in the code to do that?

[teto]

I call this in every command:
shlex.split(args)

Isn't preparse a good place to do it ? (i haven't tried yet)

def preparse(raw):
    return shlex.split(raw)

[tleonhardt]

@teto
cmd2 has the "normal" cmd2 way of doing commands, but it also has the much, much nicer way using the @options decorator. The @options decorator uses optparse to parse options, but it also automatically uses shell.split() on the args. So an @options command definition looks like the following

@options([make_option('-p', '--piglatin', action="store_true", help="atinLay"),
                  make_option('-s', '--shout', action="store_true", 
                                         help="N00B EMULATION MODE"),
                  make_option('-r', '--repeat', type="int", help="output [n] times")],
                 arg_desc='positional_arg_string')
    def do_oprint(self, arg, opts=None):
        """Print the options and argument list this options command was called with."""
        print('oprint was called with the following\n\toptions: {!r}\n'.format(opts))
        print('\targuments: {!r}'.format(arg))

Note that these commands accept two arguments instead of just the default one: arg and opts. In this case arg is a list of argument strings which has already been pre-parsed with shlex but in an intelligent manner where unecessary extra quotes have been stripped.

This is so convenient, that I use the @options decorator for ALL commands, sometimes with an empty list of options.

So the whole point of this issue is to figure out an intelligent way to have "normal" commands get parsed in a similar fashion to the @options commands in terms of that argument splitting.

I'm going to have to think about the possibility of putting some code in preparse to handle this. That might just work, but the way that @options commands interact with it would potentially be problematic. I was tentatively waiting to work on this issue until #36 gets resolved because any change there would likely radically change this.

[tleonhardt]

This issue should now be revisited in light of the new argparse-based decorator in lieu of the old optparse-based one.

Basically, it would be nice that even if for normal non-decorated commands the arguments where parsed with shlex.split() and the do_ methods get passed a list of arguments instead of a single string.

[kotfu]

I can think of several ways we could make this happen:

1. Pass do_ methods a list of arguments instead of a string. Very sys.argv-ish. If we didn't have to worry about backwards compatibility, this makes the most sense to me. But we do have to worry about backwards compatibility, so I don't this this approach is viable.
2. Use @tleonhardt's suggestion to use @options or the new @with_argument_parser decorators. They both shlex the input. If we like this method, all we have to change is the documentation. But it's not as convenient as it could be.
3. Create a decorator which you can apply to do_ methods which indicates to pass a list of arguments instead of a string. Works, but the decorator would have to be applied to every do_ method. Makes it easy to migrate to this new method, because you can migrate one method at a time.
4. Create a new subclass of str which has a arglist() method. Pass an instance of this new class to the do_ methods where we are currently passing a plain string. Existing code works fine, if you want a list of arguments, call the arglist() method.
5. Create a new attribute on the cmd() class, similar to blankLinesAllowed or case_insensitive called quoted_arguments, which defaults to False, but if set True makes cmd pass a list of arguments to do_ methods, instead of a string.

I think a combination of method 3 and method 5 would be very convenient. For new code you could use method 5. To migrate existing code, you could use method 3 during the migration. When you have changed all of your do_ methods, you use method 5 and delete all your method 3 decorators.

What do you think?

[tleonhardt]

I like the combination of methods 3 and 5 - that does sound like the most convenient option which will also preserve backward compatibility.

I originally added the USE_ARG_LIST attribute within cmd2 and associated set_use_arg_list setter function (outside of the Cmd class) for this exact purpose, but limited to within the @options decorator. Initially I had set it to False for backwards compatibility, but eventually set it to True for convenience.

We could probably repurpose that attribute now that the @options optparse-based decorator is deprecated and make the @options decorator always return arg lists.

[kotfu]

Just pushed a new branch with this change. I had to modify the onecmd() method, which means if I broke something, I broke it big. Tests pass on my local box, but I wanna see if they all pass here.

[tleonhardt]

The decorator approach works great and is a very welcome addition.

However, the modifications to onecmd() based on an attribute would create compatibility problems with the cmd module which cmd2 extends. For example, the low-level implementation of the help command actually occurs within cmd.

[kotfu]

lots more discussion and detail in the PR #249