argparse crashes in gettext when processing missing arguments

[efahl]

BPO	35785
Nosy	@efahl, @asottile, @corona10
Files	foo.mo: Binary message translation file, put in 'en_US/LC_MESSAGES/'

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-01-20.17:24:13.992>
created_at = <Date 2019-01-19.17:38:06.276>
labels = ['3.7', 'library', 'type-crash']
title = 'argparse crashes in gettext when processing missing arguments'
updated_at = <Date 2019-01-20.17:24:13.991>
user = 'https://github.com/efahl'

bugs.python.org fields:

activity = <Date 2019-01-20.17:24:13.991>
actor = 'eric.fahlgren'
assignee = 'none'
closed = True
closed_date = <Date 2019-01-20.17:24:13.992>
closer = 'eric.fahlgren'
components = ['Library (Lib)']
creation = <Date 2019-01-19.17:38:06.276>
creator = 'eric.fahlgren'
dependencies = []
files = ['48068']
hgrepos = []
issue_num = 35785
keywords = []
message_count = 7.0
messages = ['334065', '334069', '334074', '334076', '334077', '334080', '334093']
nosy_count = 4.0
nosy_names = ['bethard', 'eric.fahlgren', 'Anthony Sottile', 'corona10']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue35785'
versions = ['Python 3.7']

[efahl]

When argparse is configured with an option that takes arguments, then the script is invoked with the switch but no arguments, a nonsensical exception is raised during gettext processing.

In the 3.7.1 source, the error is at line 2077 of argparse.py, where 'action.nargs' is not an integer as expected by 'ngettext', but one of None, '*' or '?':

            default = ngettext('expected %s argument',
                               'expected %s arguments',
                               action.nargs) % action.nargs
            msg = nargs_errors.get(action.nargs, default)

Fix should be pretty trivial, swap the two lines and if 'get' produces None, only then compute the default.

File "C:\Program Files\Python37\lib\argparse.py", line 1749, in parse_args
args, argv = self.parse_known_args(args, namespace)
File "C:\Program Files\Python37\lib\argparse.py", line 1781, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
File "C:\Program Files\Python37\lib\argparse.py", line 1987, in _parse_known_args
start_index = consume_optional(start_index)
File "C:\Program Files\Python37\lib\argparse.py", line 1917, in consume_optional
arg_count = match_argument(action, selected_patterns)
File "C:\Program Files\Python37\lib\argparse.py", line 2079, in _match_argument
action.nargs) % action.nargs
File "C:\Program Files\Python37\lib\gettext.py", line 631, in ngettext
return dngettext(current_domain, msgid1, msgid2, n)
File "C:\Program Files\Python37\lib\gettext.py", line 610, in dngettext
return t.ngettext(msgid1, msgid2, n)
File "C:\Program Files\Python37\lib\gettext.py", line 462, in ngettext
tmsg = self._catalog[(msgid1, self.plural(n))]
File "<string>", line 4, in func
File "C:\Program Files\Python37\lib\gettext.py", line 168, in _as_int
(n.__class_.__name__,)) from None
TypeError: Plural value must be an integer, got NoneType

[asottile]

Can you provide a reproducer? I'm having difficulty reproducing with this script:

import argparse

p = argparse.ArgumentParser()
p.add_argument('--foo', nargs=None)
args = p.parse_args()

$ python3.7 --version
Python 3.7.2
$ python3.7 t.py --foo
usage: t.py [-h] [--foo FOO]
t.py: error: argument --foo: expected one argument

[efahl]

After a bit more digging, it's a side effect of having the locale set with 'Plural-Forms'. I've attached the resulting .mo file, but since it's a binary, I'm not sure it will work cross-platform, so here's how to recreate it.

cat en_US/LC_MESSAGES/foo.po
msgid ""
msgstr "Plural-Forms: nplurals=2; plural=(n != 1);\n"

python /Python37/Tools/i18n/msgfmt.py en_US/LC_MESSAGES/foo.po
ll en_US/LC_MESSAGES/
-rwx------+ 1 efahlgren Domain Users 89 2019-01-19 14:36 foo.mo*
-rw-r--r--+ 1 efahlgren Domain Users 69 2019-01-19 14:34 foo.po

Then you can reproduce with some setup in your script:

import os
import gettext
import argparse

os.putenv('LANG', 'en_US')  # Just to make sure.
gettext.bindtextdomain('foo', '.')
gettext.textdomain('foo')

p = argparse.ArgumentParser()
p.add_argument('--foo', nargs=None)
p.parse_args()

[corona10]

No crash at Python 3.7.2+ (heads/3.7:47290e7642, Jan 20 2019, 12:22:44)
and master branch

[corona10]

Umm looks like I should pass an argument on cmd to reproduce it. I will try to reproduce it later

[corona10]

➜ cpython.git git:(3.7) ✗ ./python.exe bpo-35785.py --foo test
➜ cpython.git git:(3.7) ✗ ./python.exe bpo-35785.py --foo 1
➜ cpython.git git:(3.7) ✗ ./python.exe bpo-35785.py --foo
usage: bpo-35785.py [-h] [--foo FOO]
bpo-35785.py: error: argument --foo: expected one argument

[efahl]

Thanks, I installed 3.7.2 on one of our non-production machines and it appears that gettext has been fixed, so I'm closing this.

python -V
Python 3.7.2
python bpo-35785.py --foo
usage: bpo-35785.py [-h] [--foo FOO]
bpo-35785.py: error: argument --foo: expected one argument