Cmd: list available completions from the cmd.Cmd subclass and filter out EOF handler(s)

[ngie-eign]

BPO	13214
Nosy	@rhettinger, @ngie-eign, @catherinedevlin, @boompig, @aldwinaldwin
PRs	bpo-13214: raise EOFError in Lib/cmd.py #17074
Files	python-cmd-better-filtering.patch python-cmd-better-filtering.patch python-cmd-better-filtering-py3.patch test_cmd.patch: expand test_cmd2.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2011-10-18.19:16:06.030>
labels = ['type-feature', 'library']
title = 'Cmd: list available completions from the cmd.Cmd subclass and filter out EOF handler(s)'
updated_at = <Date 2019-11-07.00:41:34.063>
user = 'https://github.com/ngie-eign'

bugs.python.org fields:

activity = <Date 2019-11-07.00:41:34.063>
actor = 'python-dev'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2011-10-18.19:16:06.030>
creator = 'ngie'
dependencies = []
files = ['23449', '23485', '26593', '26594']
hgrepos = []
issue_num = 13214
keywords = ['patch']
message_count = 8.0
messages = ['145856', '145868', '146060', '166859', '166861', '309788', '343874', '353374']
nosy_count = 6.0
nosy_names = ['rhettinger', 'ngie', 'Catherine.Devlin', 'boompig', 'aldwinaldwin', 'Kimball Leavitt']
pr_nums = ['17074']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue13214'
versions = ['Python 3.6']

[ngie-eign]

1. The current code in cmd.get_names does a dir on the derived class for
   cmd.Cmd object instead, which means that if I do something similar to
   the following:

class CLI(cmd.Cmd):
   def register_subcommand(self, cmd, cli_class):
       def call_cli(self):
           cli = cli_class()
           cli.cmdloop()
       setattr(self, 'do_%s' % (cmd, ), call_cli)

it won't register the command in the completion list or help list.

2. Registering a do_EOF handler is a bit of a hack to work around
   the fact that entering in ^D on the command line prints out:

   (Cmd) *** Unknown syntax: EOF

   I can't speak to the fact that my desired behavior should be
   enforced (basically trickle up the EOFError like so):

   def do_EOF(self, arg):
       raise EOFError

so what I'm proposing instead is that it be filtered out by default.

Given that there's some value in allowing developers to build custom filter rules for the completions (example: certain commands can be undocumented, hidden to lower privilege users, etc), I figured it would be wise to create a custom mechanism for filtering out commands.

This could be cleaned up to use better python idioms like a generator, cache the data and have a refresh method, etc, but this is a good first start.

[rhettinger]

This looks to be a reasonable request.

I think the patch would be better if the filtering were done directly in get_names(). A subclass can override or extend that method if it wants to customize the filter.

[ngie-eign]

Here's a version incorporating your suggestion and better documenting the choices and the method for overriding purposes. I have a few reservations with the current implementation:

1. As noted, the information for the class really could be and should be cached as the attributes of a given cmd.Cmd derived class don't change all that frequently.
2. One has to override the entire function in order to get what I consider standard functionality (filtering).. so I don't know if that's a good idea.
3. I've thought about the do_EOF handler stuff, and it would be nice if that was shoved into the completer method(s) as a keyword argument, defaulting to False -- that way one could avoid having to explicitly install an EOF handler when dealing with ^D, etc, but this can be hashed out better in a different issue, over IRC, email, etc.

This module could be better cleaned up (isn't PEP-8 compliant, overrides built-ins, is pythonic but not super pythonic, etc), but I'll see what other modules exist out there that could be used in its place, because they could have resolved some of these issues. There is some value that can be obtained from pexpect, some of the other cmd module variants, etc .. I just like this module because it's nice, simple, and standard -- it just needs a little love and it will be awesome.

Anyhow -- thanks again for the work :).

[catherinedevlin]

Needed to update the patch slightly for Python 3; now that filter() returns an iterator, do_help's call to
names = self.get_names()
followed by
names.sort()
was throwing an error, so I changed get_names to return a list.

[catherinedevlin]

Change to test_cmd.py to test for help displaying the name of the registered subcommand (as well as a simple test for the basic operation of the registered sub-CLI).

[boompig]

If you write a handler for EOF like so:

from cmd import Cmd

class FooShell(Cmd):
	def do_EOF(self, args):
		# exit on EOF
		raise SystemExit()

shell = FooShell()
shell.cmdloop()

Then when running the shell, you can see "EOF" as an undocumented command in the help screen. You can see this when typing "?".

$ python fooshell.py
(Cmd) ?

Documented commands (type help <topic>):
========================================
help

Undocumented commands:
======================
EOF

I believe the correct behaviour should be (1) don't show it in the undocumented commands, since it's not really a command; and (2) maybe create a built-in command for this, since the literal string "EOF" is also caught by this handler.

[aldwinaldwin]

The EOF mentioned in msg309788 is the first reason why I searched in the library code for a solution. Then saw it as an opportunity to create hidden functions in the shell. So I created bpo-37030 [0] and PR13536 [1]. Later got notified about this thread.

[0] https://bugs.python.org/issue37030#msg343873
[1] #13536

[KimballLeavitt]

I know that many people do something like:

def do_EOF(self, arg):
    return True

to exit the program when you press Ctrl+d. Others might prefer something like ngie https://bugs.python.org/issue13214#msg145856:

def do_EOF(self, arg):
    raise EOFError

The issue that I have is if the command you enter is 'EOF' (or 'EOF --some --arg'), you end up calling your do_EOF function. I think this unintended side effect could be avoided if the check for EOFError was removed and the exception was just raised. (see https://github.com/python/cpython/blob/master/Lib/cmd.py#L127).