New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for groups of arguments. #205

Closed
Kentzo opened this Issue Oct 10, 2013 · 28 comments

Comments

Projects
None yet
5 participants
@Kentzo

Kentzo commented Oct 10, 2013

Argparse supports grouping of arguments for better help message so cement should.

Current workaround is to override _process_arguments

@ghost ghost assigned derks Oct 22, 2013

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Oct 22, 2013

Member

Just to clarify, are you referring to sub-parsers/sub-commands? http://docs.python.org/2/library/argparse.html#sub-commands

Member

derks commented Oct 22, 2013

Just to clarify, are you referring to sub-parsers/sub-commands? http://docs.python.org/2/library/argparse.html#sub-commands

@Kentzo

This comment has been minimized.

Show comment
Hide comment
@Kentzo

Kentzo commented Oct 23, 2013

@derks Right.

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Oct 25, 2013

Member

The app.args object is an argparse 'parser'. Anything that you can do with argparse, you should be able to do with app.args without issue. Have you tried, and run into limitations? For example, you should be able to do app.args.add_subparsers(...).

Member

derks commented Oct 25, 2013

The app.args object is an argparse 'parser'. Anything that you can do with argparse, you should be able to do with app.args without issue. Have you tried, and run into limitations? For example, you should be able to do app.args.add_subparsers(...).

@Kentzo

This comment has been minimized.

Show comment
Hide comment
@Kentzo

Kentzo Oct 25, 2013

Sure if I'd use it alone I could add anything. But since I'm using a frameworks that has its own way to specify commands and options, how would I make them work together?

Kentzo commented Oct 25, 2013

Sure if I'd use it alone I could add anything. But since I'm using a frameworks that has its own way to specify commands and options, how would I make them work together?

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Oct 25, 2013

Member

You are right, basically, the default implementation of the IController interface (CementBaseController) does not use Argparse's sub-parsers to implement sub-commands. That said, it is possible to write your own BaseController... or sub-class CementBaseController and override/integrate the features you need.

CementBaseController is very complex... probably the most complex piece of Cement, and I wouldn't expect anyone to attempt to write their own unless absolutely necessary. That said, it is entirely possible.... the interface itself is very simple... the only thing that an IController implementation has to provide is a _dispatch() function. What happens is, if CementApp.Meta.base_controller is defined... then 'runtime' is passed to the controller to handle parsing arguments and dispatching commands, rather than parsing arguments itself.

The question is, what happens when run-time is passed to your IController implementation? Currently, that is all handled by CementBaseController, so you'll probably need to study that before attempting to write your own.

Member

derks commented Oct 25, 2013

You are right, basically, the default implementation of the IController interface (CementBaseController) does not use Argparse's sub-parsers to implement sub-commands. That said, it is possible to write your own BaseController... or sub-class CementBaseController and override/integrate the features you need.

CementBaseController is very complex... probably the most complex piece of Cement, and I wouldn't expect anyone to attempt to write their own unless absolutely necessary. That said, it is entirely possible.... the interface itself is very simple... the only thing that an IController implementation has to provide is a _dispatch() function. What happens is, if CementApp.Meta.base_controller is defined... then 'runtime' is passed to the controller to handle parsing arguments and dispatching commands, rather than parsing arguments itself.

The question is, what happens when run-time is passed to your IController implementation? Currently, that is all handled by CementBaseController, so you'll probably need to study that before attempting to write your own.

@Kentzo

This comment has been minimized.

Show comment
Hide comment
@Kentzo

Kentzo Oct 25, 2013

I think since the framework is built on top of argparse it should provide full set of features on its own.

Kentzo commented Oct 25, 2013

I think since the framework is built on top of argparse it should provide full set of features on its own.

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Oct 25, 2013

Member

I completely hear your point, however the framework itself does not rely on, or soley function off of Argparse. That said, the default handlers for argument parsing (ArgparseArgumentHandler) and command dispatch (CementBaseController) are indeed built on top of Argparse. Cement offers pre-built handlers that implement the required interfaces it relies on... but all of those handlers can be replaced with an alternative implementation if they do not work for you.

In my opinion, a Framework should provide a solid balance between 'out of the box' functionality that solves 80% of users needs, while being flexible enough to allow developers to solve for the other 20% should they need to. I would be more than happy to evaluate an alternative implementation of the IController interface if you submit a pull request with the code.

Thanks for the feedback!

Member

derks commented Oct 25, 2013

I completely hear your point, however the framework itself does not rely on, or soley function off of Argparse. That said, the default handlers for argument parsing (ArgparseArgumentHandler) and command dispatch (CementBaseController) are indeed built on top of Argparse. Cement offers pre-built handlers that implement the required interfaces it relies on... but all of those handlers can be replaced with an alternative implementation if they do not work for you.

In my opinion, a Framework should provide a solid balance between 'out of the box' functionality that solves 80% of users needs, while being flexible enough to allow developers to solve for the other 20% should they need to. I would be more than happy to evaluate an alternative implementation of the IController interface if you submit a pull request with the code.

Thanks for the feedback!

@Kentzo

This comment has been minimized.

Show comment
Hide comment
@Kentzo

Kentzo Oct 26, 2013

@derks I agree with your point of 80/20 but in case of Cement it seems like that 20% is a lot of work.

I decided to stick with another framework for now which is more flexible, so I don't have time to send a PR. Please keep this issue open though, probably someone else will find and fix it or maybe I'll choose cement for my next cmd tool and therefore do that myself.

Kentzo commented Oct 26, 2013

@derks I agree with your point of 80/20 but in case of Cement it seems like that 20% is a lot of work.

I decided to stick with another framework for now which is more flexible, so I don't have time to send a PR. Please keep this issue open though, probably someone else will find and fix it or maybe I'll choose cement for my next cmd tool and therefore do that myself.

@mahmoudimus

This comment has been minimized.

Show comment
Hide comment
@mahmoudimus

mahmoudimus Apr 21, 2014

+1 on this topic. @derks - any progress here?

mahmoudimus commented Apr 21, 2014

+1 on this topic. @derks - any progress here?

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Apr 21, 2014

Member

@mahmoudimus no progress has been done here. For any kind of development to happen on this, I'd really need a clearer picture of what you (or anyone else with this issue) is trying to accomplish and why/how Cement stands in the way on that.

Always open to implementing features, I just need it broken down better. What exactly are you looking for Cement to do/provide? Any feedback is appreciated!

Thanks.

Member

derks commented Apr 21, 2014

@mahmoudimus no progress has been done here. For any kind of development to happen on this, I'd really need a clearer picture of what you (or anyone else with this issue) is trying to accomplish and why/how Cement stands in the way on that.

Always open to implementing features, I just need it broken down better. What exactly are you looking for Cement to do/provide? Any feedback is appreciated!

Thanks.

@mahmoudimus

This comment has been minimized.

Show comment
Hide comment
@mahmoudimus

mahmoudimus Apr 21, 2014

@derks Here's a sample gist: https://gist.github.com/mahmoudimus/11159920

https://gist.github.com/mahmoudimus/11159920#file-vizor-py-L38-L40 How do we dry this up? This is why argparse subcommands works well here, because it can inherit from a root parser.

I don't see how I can accomplish this from: http://builtoncement.com/2.2/examples/abstract_base_controllers.html?highlight=examples

mahmoudimus commented Apr 21, 2014

@derks Here's a sample gist: https://gist.github.com/mahmoudimus/11159920

https://gist.github.com/mahmoudimus/11159920#file-vizor-py-L38-L40 How do we dry this up? This is why argparse subcommands works well here, because it can inherit from a root parser.

I don't see how I can accomplish this from: http://builtoncement.com/2.2/examples/abstract_base_controllers.html?highlight=examples

@mahmoudimus

This comment has been minimized.

Show comment
Hide comment
@mahmoudimus

mahmoudimus Apr 21, 2014

What I want to do is the cmdline you see above: vizor scanner should print w/ error message 2 say "you need one of aws|file"

Handling that is very complicated the way cement works due to the way usage is processed.

mahmoudimus commented Apr 21, 2014

What I want to do is the cmdline you see above: vizor scanner should print w/ error message 2 say "you need one of aws|file"

Handling that is very complicated the way cement works due to the way usage is processed.

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Apr 21, 2014

Member

Currently, there are two ways to approach this ... the way you currently have it (recommended), with stacked/nested sub-controllers ... OR 'aws' and 'file' could be passed as an optional command line argument added to scanner's arguments, that the scanner controller processes to determine how to proceed. In the second case, --ec2-credentials would have to be added to scanner so that probably isn't ideal... but it is an option.

In your current example, my approach would be to put that expected validation (i.e. "A sub-command is required") into the default() function of scanner. Therefore, if no 'aws' or 'file' is passed (as a sub-command) then you get the error from default().

I see the value in argparse subcommands, and re-working how Cement handles this to support that. That said, CementBaseController is one of the most complex pieces of the framework. It will time some time to re-work it however I will dedicate some time to exploring Argparse/CementBaseController this week to see if I can work up an alternative ... CementBaseController2 or something that might accomplish what you're looking for.

Member

derks commented Apr 21, 2014

Currently, there are two ways to approach this ... the way you currently have it (recommended), with stacked/nested sub-controllers ... OR 'aws' and 'file' could be passed as an optional command line argument added to scanner's arguments, that the scanner controller processes to determine how to proceed. In the second case, --ec2-credentials would have to be added to scanner so that probably isn't ideal... but it is an option.

In your current example, my approach would be to put that expected validation (i.e. "A sub-command is required") into the default() function of scanner. Therefore, if no 'aws' or 'file' is passed (as a sub-command) then you get the error from default().

I see the value in argparse subcommands, and re-working how Cement handles this to support that. That said, CementBaseController is one of the most complex pieces of the framework. It will time some time to re-work it however I will dedicate some time to exploring Argparse/CementBaseController this week to see if I can work up an alternative ... CementBaseController2 or something that might accomplish what you're looking for.

@mahmoudimus

This comment has been minimized.

Show comment
Hide comment
@mahmoudimus

mahmoudimus Apr 22, 2014

@derks - re: second approach, yep that's what I figured and it's less than ideal, which is why I am +1 on this issue.

Re; your validation, that's fair. But now, I think you understand what I wanted to get out of Cement w/o having to go through argparser directly.

mahmoudimus commented Apr 22, 2014

@derks - re: second approach, yep that's what I figured and it's less than ideal, which is why I am +1 on this issue.

Re; your validation, that's fair. But now, I think you understand what I wanted to get out of Cement w/o having to go through argparser directly.

@derks derks removed dev/2.3.x labels Aug 21, 2014

@derks derks self-assigned this Aug 21, 2014

@derks derks added the 1 - Ready label Feb 25, 2015

@derks derks removed the 1 - Ready label May 6, 2015

derks added a commit that referenced this issue Jun 2, 2015

derks added a commit that referenced this issue Jun 2, 2015

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 2, 2015

Member

I have finally (finally!) put a lot of time into refactoring CementBaseController to use Argparse in a more natural way. Currently, this work lives in a feature branch called argument_handling and would love some feedback on it if any of you all have time.

Essentially, this has completely reworked how commands and arguments are collected from controllers and passed to Argparse. There are a number of features including:

Processing of All Arguments, From All Controllers

The current way with CementBaseController looks like:

$ myapp [sub-command] {options}

$ myapp [nested-controller] [sub-command] {options}

$ myapp [nested-controller] [nested-controller] [sub-command] {options}

The final/last controller would have to have all the arguments attached to it, and you can not pass options to any of the previously parsed namespaces. That said, the new way looks like:

$ myapp {options}

$ myapp {options} [sub-command] {options}

$ myapp {options} [nested-controller] {options} [sub-command] {options}

For example:

$ myapp third --third-foo "3" command-3 -C "KAPLA"
Inside Third.command_3()
Got Big C Option > KAPLA
Got Third Foo Option > 3

Controller Arguments vs. Command Arguments

With ArgparseController we now have the ability to add additional arguments to the sub-command and controller separates. For example:

from cement.ext.ext_argparse import ArgparseController, expose

class MyController(ArgparseController):
    class Meta:
        label = 'my_controller'
        arguments = [
           ( ['-f', '--foo'], dict(help='foo option help') ),
        ]

    @expose(
        arguments=[
            ( ['--other-foo'], dict(help='option under my-sub-command) ),
        ],
        help='my-sub-command help',
        # other options passed directly to argparse
    )
    def my_sub_command(self):
        if self.app.pargs.other_foo:
            print('Received Other Foo Option > %s' % self.app.pargs.other_foo)

Essentially, all **kw passed to @expose will be handled off to Argparse, give flexibility to how arguments are handled (controller level vs. comand level).

Working CementApp Example

from cement.core.foundation import CementApp
from cement.ext.ext_argparse import ArgparseController, expose

class Base(ArgparseController):
    class Meta:
        label = 'base'
        description = 'MyApp Does Amazing Things'
        arguments = [
            ( ['-f', '--foo'], dict(help='the notorious foo option')),
            ]

    @expose(hide=True)
    def default(self):
        print("Inside Base.default()")

    @expose(help='command-1 help message')
    def command_1(self):
        print("Inside Base.command_1()")

    def _process_parsed_arguments(self):
        if hasattr(self.app.pargs, 'foo') and self.app.pargs.foo:
            print('Got Foo Option > %s' % self.app.pargs.foo)

class Second(ArgparseController):
    class Meta:
        label = 'second'
        stacked_on = 'base'
        stacked_type = 'embedded'
        arguments = [
            (['--second-foo'], dict(help='second foo option')),
            ]

    @expose(help='command-2 help message')
    def command_2(self):
        print("Inside Second.command_2()")

class Third(ArgparseController):
    class Meta:
        label = 'third'
        stacked_on = 'base'
        stacked_type = 'nested'
        arguments = [
            (['--third-foo'], dict(help='third foo option', dest='third_foo')),
            ]

    @expose(
        arguments=[
            (['-C'], dict(help='the big C option', dest='big_c'))
            ],
        help='command-3 help message', 
        )
    def command_3(self):
        print("Inside Third.command_3()")

        if self.app.pargs.big_c:
            print('Got Big C Option > %s' % self.app.pargs.big_c)

    @expose(hide=True, help='asfasfasdf')
    def default(self):
        print("Inside Third.default()")

    def _process_parsed_arguments(self):
        if hasattr(self.app.pargs, 'third_foo') and self.app.pargs.third_foo:
            print('Got Third Foo Option > %s' % self.app.pargs.third_foo)


class MyApp(CementApp):
    class Meta:
        label = 'myapp'
        handlers = [Base, Second, Third]

def main():
    with MyApp() as app:
        app.run()

if __name__ == '__main__':
    main()

Looks like:

usage: myapp [-h] [--debug] [--quiet] [-f FOO] [--second-foo SECOND_FOO]
             {third,command-1,default,command-2} ...

optional arguments:
  -h, --help            show this help message and exit
  --debug               toggle debug output
  --quiet               suppress all output
  -f FOO, --foo FOO     the notorious foo option
  --second-foo SECOND_FOO
                        second foo option

sub-commands:
  {third,command-1,default,command-2}
    third               third controller
    command-1           command-1 help message
    command-2           command-2 help message


$ python myapp.py
Inside Base.default()


$ python myapp.py -f foo-1 command-1
Got Foo Option > foo-1
Inside Base.command_1()



$ python myapp.py third --help
usage: myapp third [-h] [--third-foo THIRD_FOO] {command-3,default} ...

optional arguments:
  -h, --help            show this help message and exit
  --third-foo THIRD_FOO
                        third foo option

sub-commands:
  {command-3,default}
    command-3           command-3 help message


$ python myapp.py -f foo-1 third --third-foo foo-3 command-3 -C KAPLA
Got Foo Option > foo-1
Got Third Foo Option > foo-3
Inside Third.command_3()
Got Big C Option > KAPLA

Note: Only testing on Python 3.

Member

derks commented Jun 2, 2015

I have finally (finally!) put a lot of time into refactoring CementBaseController to use Argparse in a more natural way. Currently, this work lives in a feature branch called argument_handling and would love some feedback on it if any of you all have time.

Essentially, this has completely reworked how commands and arguments are collected from controllers and passed to Argparse. There are a number of features including:

Processing of All Arguments, From All Controllers

The current way with CementBaseController looks like:

$ myapp [sub-command] {options}

$ myapp [nested-controller] [sub-command] {options}

$ myapp [nested-controller] [nested-controller] [sub-command] {options}

The final/last controller would have to have all the arguments attached to it, and you can not pass options to any of the previously parsed namespaces. That said, the new way looks like:

$ myapp {options}

$ myapp {options} [sub-command] {options}

$ myapp {options} [nested-controller] {options} [sub-command] {options}

For example:

$ myapp third --third-foo "3" command-3 -C "KAPLA"
Inside Third.command_3()
Got Big C Option > KAPLA
Got Third Foo Option > 3

Controller Arguments vs. Command Arguments

With ArgparseController we now have the ability to add additional arguments to the sub-command and controller separates. For example:

from cement.ext.ext_argparse import ArgparseController, expose

class MyController(ArgparseController):
    class Meta:
        label = 'my_controller'
        arguments = [
           ( ['-f', '--foo'], dict(help='foo option help') ),
        ]

    @expose(
        arguments=[
            ( ['--other-foo'], dict(help='option under my-sub-command) ),
        ],
        help='my-sub-command help',
        # other options passed directly to argparse
    )
    def my_sub_command(self):
        if self.app.pargs.other_foo:
            print('Received Other Foo Option > %s' % self.app.pargs.other_foo)

Essentially, all **kw passed to @expose will be handled off to Argparse, give flexibility to how arguments are handled (controller level vs. comand level).

Working CementApp Example

from cement.core.foundation import CementApp
from cement.ext.ext_argparse import ArgparseController, expose

class Base(ArgparseController):
    class Meta:
        label = 'base'
        description = 'MyApp Does Amazing Things'
        arguments = [
            ( ['-f', '--foo'], dict(help='the notorious foo option')),
            ]

    @expose(hide=True)
    def default(self):
        print("Inside Base.default()")

    @expose(help='command-1 help message')
    def command_1(self):
        print("Inside Base.command_1()")

    def _process_parsed_arguments(self):
        if hasattr(self.app.pargs, 'foo') and self.app.pargs.foo:
            print('Got Foo Option > %s' % self.app.pargs.foo)

class Second(ArgparseController):
    class Meta:
        label = 'second'
        stacked_on = 'base'
        stacked_type = 'embedded'
        arguments = [
            (['--second-foo'], dict(help='second foo option')),
            ]

    @expose(help='command-2 help message')
    def command_2(self):
        print("Inside Second.command_2()")

class Third(ArgparseController):
    class Meta:
        label = 'third'
        stacked_on = 'base'
        stacked_type = 'nested'
        arguments = [
            (['--third-foo'], dict(help='third foo option', dest='third_foo')),
            ]

    @expose(
        arguments=[
            (['-C'], dict(help='the big C option', dest='big_c'))
            ],
        help='command-3 help message', 
        )
    def command_3(self):
        print("Inside Third.command_3()")

        if self.app.pargs.big_c:
            print('Got Big C Option > %s' % self.app.pargs.big_c)

    @expose(hide=True, help='asfasfasdf')
    def default(self):
        print("Inside Third.default()")

    def _process_parsed_arguments(self):
        if hasattr(self.app.pargs, 'third_foo') and self.app.pargs.third_foo:
            print('Got Third Foo Option > %s' % self.app.pargs.third_foo)


class MyApp(CementApp):
    class Meta:
        label = 'myapp'
        handlers = [Base, Second, Third]

def main():
    with MyApp() as app:
        app.run()

if __name__ == '__main__':
    main()

Looks like:

usage: myapp [-h] [--debug] [--quiet] [-f FOO] [--second-foo SECOND_FOO]
             {third,command-1,default,command-2} ...

optional arguments:
  -h, --help            show this help message and exit
  --debug               toggle debug output
  --quiet               suppress all output
  -f FOO, --foo FOO     the notorious foo option
  --second-foo SECOND_FOO
                        second foo option

sub-commands:
  {third,command-1,default,command-2}
    third               third controller
    command-1           command-1 help message
    command-2           command-2 help message


$ python myapp.py
Inside Base.default()


$ python myapp.py -f foo-1 command-1
Got Foo Option > foo-1
Inside Base.command_1()



$ python myapp.py third --help
usage: myapp third [-h] [--third-foo THIRD_FOO] {command-3,default} ...

optional arguments:
  -h, --help            show this help message and exit
  --third-foo THIRD_FOO
                        third foo option

sub-commands:
  {command-3,default}
    command-3           command-3 help message


$ python myapp.py -f foo-1 third --third-foo foo-3 command-3 -C KAPLA
Got Foo Option > foo-1
Got Third Foo Option > foo-3
Inside Third.command_3()
Got Big C Option > KAPLA

Note: Only testing on Python 3.

@derks derks added this to the 2.7.2 Development milestone Jun 2, 2015

@derks derks added the stable/2.10.x label Jun 2, 2015

@dano

This comment has been minimized.

Show comment
Hide comment
@dano

dano Jun 8, 2015

@derks I actually just came to the tracker to request the ability to provide command-specific arguments. I'll take the argument_handling branch for a spin in the next couple of days and let you know how it goes.

dano commented Jun 8, 2015

@derks I actually just came to the tracker to request the ability to provide command-specific arguments. I'll take the argument_handling branch for a spin in the next couple of days and let you know how it goes.

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 10, 2015

Member

Thanks @dano! Let me know what you think... any feedback is appreciated.

Member

derks commented Jun 10, 2015

Thanks @dano! Let me know what you think... any feedback is appreciated.

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 10, 2015

Member

Note to self, this section can probably be refactored:

self._controllers already resolved nesting/embedding order which (I believe) eliminates the need for all the whacky shit because self._controllers should be iterable in its current order.

Member

derks commented Jun 10, 2015

Note to self, this section can probably be refactored:

self._controllers already resolved nesting/embedding order which (I believe) eliminates the need for all the whacky shit because self._controllers should be iterable in its current order.

@dano

This comment has been minimized.

Show comment
Hide comment
@dano

dano Jun 10, 2015

@derks I'm hitting errors when I try to use your test script (using Python 3.4):

dan@dan-002:~$ python3 myapp.py
Inside Base.default()
dan@dan002:~$ python3 myapp.py -f foo-1 command-1
Traceback (most recent call last):
  File "myapp.py", line 78, in <module>
    main()
  File "myapp.py", line 75, in main
    app.run()
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/core/foundation.py", line 764, in run
    self.controller._dispatch()
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/ext/ext_argparse.py", line 596, in _dispatch
    self.app._parse_args()
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/core/foundation.py", line 975, in _parse_args
    self._parsed_args = self.args.parse(self.argv)
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/ext/ext_argparse.py", line 55, in parse
    return self.parse_args(arg_list)
  File "/usr/lib/python3.4/argparse.py", line 1737, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.4/argparse.py", line 1769, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.4/argparse.py", line 1978, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/lib/python3.4/argparse.py", line 1934, in consume_positionals
    take_action(action, args)
  File "/usr/lib/python3.4/argparse.py", line 1843, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/lib/python3.4/argparse.py", line 1138, in __call__
    subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
  File "/usr/lib/python3.4/argparse.py", line 1769, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.4/argparse.py", line 1978, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/lib/python3.4/argparse.py", line 1934, in consume_positionals
    take_action(action, args)
  File "/usr/lib/python3.4/argparse.py", line 1843, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/ext/ext_argparse.py", line 523, in __call__
    func_name = _clean_command_func(namespace.command)
AttributeError: 'Namespace' object has no attribute 'command'

dano commented Jun 10, 2015

@derks I'm hitting errors when I try to use your test script (using Python 3.4):

dan@dan-002:~$ python3 myapp.py
Inside Base.default()
dan@dan002:~$ python3 myapp.py -f foo-1 command-1
Traceback (most recent call last):
  File "myapp.py", line 78, in <module>
    main()
  File "myapp.py", line 75, in main
    app.run()
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/core/foundation.py", line 764, in run
    self.controller._dispatch()
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/ext/ext_argparse.py", line 596, in _dispatch
    self.app._parse_args()
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/core/foundation.py", line 975, in _parse_args
    self._parsed_args = self.args.parse(self.argv)
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/ext/ext_argparse.py", line 55, in parse
    return self.parse_args(arg_list)
  File "/usr/lib/python3.4/argparse.py", line 1737, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.4/argparse.py", line 1769, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.4/argparse.py", line 1978, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/lib/python3.4/argparse.py", line 1934, in consume_positionals
    take_action(action, args)
  File "/usr/lib/python3.4/argparse.py", line 1843, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/lib/python3.4/argparse.py", line 1138, in __call__
    subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
  File "/usr/lib/python3.4/argparse.py", line 1769, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.4/argparse.py", line 1978, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/lib/python3.4/argparse.py", line 1934, in consume_positionals
    take_action(action, args)
  File "/usr/lib/python3.4/argparse.py", line 1843, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/local/lib/python3.4/dist-packages/cement-2.7.1.dev20150602060238-py3.4.egg/cement/ext/ext_argparse.py", line 523, in __call__
    func_name = _clean_command_func(namespace.command)
AttributeError: 'Namespace' object has no attribute 'command'
@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 10, 2015

Member

@dano Can I see your code?

Member

derks commented Jun 10, 2015

@dano Can I see your code?

@dano

This comment has been minimized.

Show comment
Hide comment
@dano

dano Jun 10, 2015

@derks I'm just running your "Working CementApp Example" without any changes. I pulled down the argument_handling branch this morning and installed that. If it's working for you, I'm not sure what the issue might be...

dano commented Jun 10, 2015

@derks I'm just running your "Working CementApp Example" without any changes. I pulled down the argument_handling branch this morning and installed that. If it's working for you, I'm not sure what the issue might be...

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 10, 2015

Member

@dano Thanks... let me try it out again.

Member

derks commented Jun 10, 2015

@dano Thanks... let me try it out again.

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 10, 2015

Member

Verified on Python 3.4.x ... doesn't have this issue on Python 3.3.x. Will hopefully push a fix shortly.

Member

derks commented Jun 10, 2015

Verified on Python 3.4.x ... doesn't have this issue on Python 3.3.x. Will hopefully push a fix shortly.

@dano

This comment has been minimized.

Show comment
Hide comment
@dano

dano Jun 10, 2015

@derks I had a feeling that was the cause...thanks for the quick triage!

dano commented Jun 10, 2015

@derks I had a feeling that was the cause...thanks for the quick triage!

derks added a commit that referenced this issue Jun 10, 2015

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 10, 2015

Member

@dano I've pushed up some changes to the argument_handling branch. Tested on Python 2.7, 3.2, and 3.4. Unfortunately, due to how Python <3.4 (Argparse) handles sub-parsers, this changes loses the 'default' command functionality. Not sure if it will be possible to get that back or not but will continue to work on it.

Please let me know how things go testing that with Python 3.4.

$ python -V
Python 3.4.3

$ python myapp.py --foo bar third command-3 -C asfas
Got Foo Option > bar
Inside Third.command_3()
Got Big C Option > asfas
Member

derks commented Jun 10, 2015

@dano I've pushed up some changes to the argument_handling branch. Tested on Python 2.7, 3.2, and 3.4. Unfortunately, due to how Python <3.4 (Argparse) handles sub-parsers, this changes loses the 'default' command functionality. Not sure if it will be possible to get that back or not but will continue to work on it.

Please let me know how things go testing that with Python 3.4.

$ python -V
Python 3.4.3

$ python myapp.py --foo bar third command-3 -C asfas
Got Foo Option > bar
Inside Third.command_3()
Got Big C Option > asfas
@dano

This comment has been minimized.

Show comment
Hide comment
@dano

dano Jun 11, 2015

@derks Thanks, that fixed it.

dano commented Jun 11, 2015

@derks Thanks, that fixed it.

derks added a commit that referenced this issue Jun 15, 2015

derks added a commit that referenced this issue Jun 17, 2015

@derks derks closed this in fea56d6 Jun 18, 2015

@derks

This comment has been minimized.

Show comment
Hide comment
@derks

derks Jun 18, 2015

Member

I have been reading more up on argument groups in Argparse, and though this new controller does not directly interact with that it does provide the means to do so.

from cement.ext.ext_argparse import ArgparseController, expose

class MyController(ArgparseController):
    class Meta:
        label = 'base'

        # add simple argument the same old way
        arguments = [
            (['-f', '--foo'], dict(help='my foo option', dest='foo'),
        ]

    def _pre_argument_parsing(self):
        group1 = self.parser.add_argument_group('my-special-group')
        group1.add_argument('--g1', help='my g1 option under my-special-group', dest='g1')
        group1.add_argument('--g2', help='my g2 option under my-special-group', dest='g2')

        # or mutually exclusive groups
        meg = self.parser.add_mutually_exclusive_group()
        meg.add_argument('--g3', help='my g3 option')
        meg.add_argument('--g4', help='my g4 option')

The above is untested example, but essentially in _pre_argument_parsing you can have full access to the sub parser for that controller... and do anything you would normally do with Argparse.

That said, there is currently no way to do this with controller sub-command arguments. in the future, it would be nice to have the framework support groups better, but this will atleast make it possible.

Member

derks commented Jun 18, 2015

I have been reading more up on argument groups in Argparse, and though this new controller does not directly interact with that it does provide the means to do so.

from cement.ext.ext_argparse import ArgparseController, expose

class MyController(ArgparseController):
    class Meta:
        label = 'base'

        # add simple argument the same old way
        arguments = [
            (['-f', '--foo'], dict(help='my foo option', dest='foo'),
        ]

    def _pre_argument_parsing(self):
        group1 = self.parser.add_argument_group('my-special-group')
        group1.add_argument('--g1', help='my g1 option under my-special-group', dest='g1')
        group1.add_argument('--g2', help='my g2 option under my-special-group', dest='g2')

        # or mutually exclusive groups
        meg = self.parser.add_mutually_exclusive_group()
        meg.add_argument('--g3', help='my g3 option')
        meg.add_argument('--g4', help='my g4 option')

The above is untested example, but essentially in _pre_argument_parsing you can have full access to the sub parser for that controller... and do anything you would normally do with Argparse.

That said, there is currently no way to do this with controller sub-command arguments. in the future, it would be nice to have the framework support groups better, but this will atleast make it possible.

@barta0

This comment has been minimized.

Show comment
Hide comment
@barta0

barta0 Nov 25, 2015

In Cement 2.6.2 I use the following workaround in the case of mutex groups:

class FooController(CementBaseController):

    class Meta:
        label = 'foo'
        stacked_on = 'base'
        stacked_type = 'nested'
        mutex_groups = [
            {
                'required': True, # or False
                'arguments': [
                    (['--option1'], dict(action='store_true')),
                    (['--option2'], dict(action='store_true'))
                ]
            },
        ]

    @expose(hide=True)
    def default(self):
        pass

As you can see I added a mutex_groups property to FooController.Meta and I define mutex groups in a similar way, as we define arguments in Cement.

This property is processed by this function:

def set_mutex_groups_hook(app):
    if hasattr(app.controller._dispatch_command['controller'].Meta, 'mutex_groups'):
        for group in app.controller._dispatch_command['controller'].Meta.mutex_groups:
            mutex_group = app.args.add_mutually_exclusive_group(required=group['required'])

            for argument in group['arguments']:
                mutex_group.add_argument(argument[0][0], **argument[1])

This function is registered as a pre_argument_parsinghook. It access the app.argswhich is an argparse.ArgumentParser instance and creates the defined mutex groups.

barta0 commented Nov 25, 2015

In Cement 2.6.2 I use the following workaround in the case of mutex groups:

class FooController(CementBaseController):

    class Meta:
        label = 'foo'
        stacked_on = 'base'
        stacked_type = 'nested'
        mutex_groups = [
            {
                'required': True, # or False
                'arguments': [
                    (['--option1'], dict(action='store_true')),
                    (['--option2'], dict(action='store_true'))
                ]
            },
        ]

    @expose(hide=True)
    def default(self):
        pass

As you can see I added a mutex_groups property to FooController.Meta and I define mutex groups in a similar way, as we define arguments in Cement.

This property is processed by this function:

def set_mutex_groups_hook(app):
    if hasattr(app.controller._dispatch_command['controller'].Meta, 'mutex_groups'):
        for group in app.controller._dispatch_command['controller'].Meta.mutex_groups:
            mutex_group = app.args.add_mutually_exclusive_group(required=group['required'])

            for argument in group['arguments']:
                mutex_group.add_argument(argument[0][0], **argument[1])

This function is registered as a pre_argument_parsinghook. It access the app.argswhich is an argparse.ArgumentParser instance and creates the defined mutex groups.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment