Tests fail on python-3.9.7 (3.9.6 worked)

[trofi]

Was discovered at NixOS/nixpkgs#136254 (comment) on NixOS after python update from 3.9.6 to 3.9.7. Two tests started failing. Posting full build log:

Sourcing pytest-remove-bytecode-hook.sh
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing setuptools-check-hook
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/p29gdljymj7bbb3dqwm0i21yp77hhi4q-argh-0.26.2.tar.gz
source root is argh-0.26.2
setting SOURCE_DATE_EPOCH to timestamp 1463000093 of file argh-0.26.2/setup.cfg
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "configurePhase" }
configuring
no configure script, doing nothing
@nix { "action": "setPhase", "phase": "buildPhase" }
building
Executing setuptoolsBuildPhase
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/argh
copying argh/interaction.py -> build/lib/argh
copying argh/utils.py -> build/lib/argh
copying argh/assembling.py -> build/lib/argh
copying argh/io.py -> build/lib/argh
copying argh/helpers.py -> build/lib/argh
copying argh/dispatching.py -> build/lib/argh
copying argh/compat.py -> build/lib/argh
copying argh/constants.py -> build/lib/argh
copying argh/__init__.py -> build/lib/argh
copying argh/decorators.py -> build/lib/argh
copying argh/exceptions.py -> build/lib/argh
copying argh/completion.py -> build/lib/argh
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/completion.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/exceptions.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/decorators.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/__init__.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/constants.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/compat.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/dispatching.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/helpers.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/io.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/assembling.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/utils.py -> build/bdist.linux-x86_64/wheel/argh
copying build/lib/argh/interaction.py -> build/bdist.linux-x86_64/wheel/argh
running install_egg_info
running egg_info
writing argh.egg-info/PKG-INFO
writing dependency_links to argh.egg-info/dependency_links.txt
writing top-level names to argh.egg-info/top_level.txt
reading manifest file 'argh.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'argh.egg-info/SOURCES.txt'
Copying argh.egg-info to build/bdist.linux-x86_64/wheel/argh-0.26.2-py3.9.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/argh-0.26.2.dist-info/WHEEL
creating 'dist/argh-0.26.2-py2.py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'argh/__init__.py'
adding 'argh/assembling.py'
adding 'argh/compat.py'
adding 'argh/completion.py'
adding 'argh/constants.py'
adding 'argh/decorators.py'
adding 'argh/dispatching.py'
adding 'argh/exceptions.py'
adding 'argh/helpers.py'
adding 'argh/interaction.py'
adding 'argh/io.py'
adding 'argh/utils.py'
adding 'argh-0.26.2.dist-info/METADATA'
adding 'argh-0.26.2.dist-info/WHEEL'
adding 'argh-0.26.2.dist-info/top_level.txt'
adding 'argh-0.26.2.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Finished executing setuptoolsBuildPhase
@nix { "action": "setPhase", "phase": "installPhase" }
installing
Executing pipInstallPhase
/build/argh-0.26.2/dist /build/argh-0.26.2
Processing ./argh-0.26.2-py2.py3-none-any.whl
Installing collected packages: argh
Successfully installed argh-0.26.2
/build/argh-0.26.2
Finished executing pipInstallPhase
@nix { "action": "setPhase", "phase": "fixupPhase" }
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/38kpzclbjgdnbxgzr2m73bq9nj8j2270-python3.9-argh-0.26.2
strip is /nix/store/p8cwgxcxqqg57kr66f3m81p2fyalnjkk-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/38kpzclbjgdnbxgzr2m73bq9nj8j2270-python3.9-argh-0.26.2/lib
patching script interpreter paths in /nix/store/38kpzclbjgdnbxgzr2m73bq9nj8j2270-python3.9-argh-0.26.2
checking for references to /build/ in /nix/store/38kpzclbjgdnbxgzr2m73bq9nj8j2270-python3.9-argh-0.26.2...
Executing pythonRemoveTestsDir
Finished executing pythonRemoveTestsDir
@nix { "action": "setPhase", "phase": "installCheckPhase" }
running install tests
============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /build/argh-0.26.2
collected 81 items

test/test_assembling.py .....ss.........                                 [ 19%]
test/test_compat.py .                                                    [ 20%]
test/test_decorators.py ......                                           [ 28%]
test/test_dispatching.py ..                                              [ 30%]
test/test_integration.py x.......xxx.............F.....F................ [ 88%]
                                                                         [ 88%]
test/test_interaction.py ...                                             [ 92%]
test/test_regressions.py ......                                          [100%]

=================================== FAILURES ===================================
_____________________________ test_invalid_choice ______________________________

    def test_invalid_choice():
        def cmd(args):
            return 1

        # root level command

        p = DebugArghParser()
        p.add_commands([cmd])

>       assert run(p, 'bar', exit=True).startswith('invalid choice')
E       assert False
E        +  where False = <built-in method startswith of str object at 0x7fffe89b22d0>('invalid choice')
E        +    where <built-in method startswith of str object at 0x7fffe89b22d0> = "argument {cmd}: invalid choice: 'bar' (choose from 'cmd')".startswith
E        +      where "argument {cmd}: invalid choice: 'bar' (choose from 'cmd')" = run(DebugArghParser(prog='py.test', usage=None, description=None, formatter_class=<class 'argh.constants.CustomFormatter'>, conflict_handler='error', add_help=True), 'bar', exit=True)

test/test_integration.py:380: AssertionError
____________________________ test_explicit_cmd_name ____________________________

    def test_explicit_cmd_name():

        @argh.named('new-name')
        def orig_name():
            return 'ok'

        p = DebugArghParser()
        p.add_commands([orig_name])
>       assert run(p, 'orig-name', exit=True).startswith('invalid choice')
E       assert False
E        +  where False = <built-in method startswith of str object at 0x7fffe89a6db0>('invalid choice')
E        +    where <built-in method startswith of str object at 0x7fffe89a6db0> = "argument {new-name}: invalid choice: 'orig-name' (choose from 'new-name')".startswith
E        +      where "argument {new-name}: invalid choice: 'orig-name' (choose from 'new-name')" = run(DebugArghParser(prog='py.test', usage=None, description=None, formatter_class=<class 'argh.constants.CustomFormatter'>, conflict_handler='error', add_help=True), 'orig-name', exit=True)

test/test_integration.py:514: AssertionError
=========================== short test summary info ============================
FAILED test/test_integration.py::test_invalid_choice - assert False
FAILED test/test_integration.py::test_explicit_cmd_name - assert False
============== 2 failed, 73 passed, 2 skipped, 4 xfailed in 0.21s ==============

[mweinelt]

Two more after upgrading to python 3.9.8.

=================================== FAILURES ===================================
__________ test_add_subparsers_when_default_command_exists__supported __________

    @pytest.mark.skipif(sys.version_info < (3,4), reason='supported since Python 3.4')
    def test_add_subparsers_when_default_command_exists__supported():

        def one(): return 1
        def two(): return 2

        p = argh.ArghParser()
        p.set_default_command(one)
        p.add_commands([two])

        ns_one = p.parse_args([])
        ns_two = p.parse_args(['two'])

        assert ns_one.get_function() == one
>       assert ns_two.get_function() == two
E       assert <function test_add_subparsers_when_default_command_exists__supported.<locals>.one at 0x7fffe89b6310> == <function test_add_subparsers_when_default_command_exists__supported.<locals>.two at 0x7fffe89b6280>
E        +  where <function test_add_subparsers_when_default_command_exists__supported.<locals>.one at 0x7fffe89b6310> = <bound method ArghNamespace.get_function of ArghNamespace(_functions_stack=[<function test_add_subparsers_when_default...x7fffe89b6280>, <function test_add_subparsers_when_default_command_exists__supported.<locals>.one at 0x7fffe89b6310>])>()
E        +    where <bound method ArghNamespace.get_function of ArghNamespace(_functions_stack=[<function test_add_subparsers_when_default...x7fffe89b6280>, <function test_add_subparsers_when_default_command_exists__supported.<locals>.one at 0x7fffe89b6310>])> = ArghNamespace(_functions_stack=[<function test_add_subparsers_when_default_command_exists__supported.<locals>.two at 0x7fffe89b6280>, <function test_add_subparsers_when_default_command_exists__supported.<locals>.one at 0x7fffe89b6310>]).get_function

test/test_assembling.py:136: AssertionError
__________ test_set_default_command_when_subparsers_exist__supported ___________

    @pytest.mark.skipif(sys.version_info < (3,4), reason='supported since Python 3.4')
    def test_set_default_command_when_subparsers_exist__supported():
        def one(): return 1
        def two(): return 2

        p = argh.ArghParser()
        p.add_commands([one])
        p.set_default_command(two)

        ns_two = p.parse_args([])
        ns_one = p.parse_args(['one'])

>       assert ns_one.get_function() == one
E       assert <function test_set_default_command_when_subparsers_exist__supported.<locals>.two at 0x7fffe8d7fb80> == <function test_set_default_command_when_subparsers_exist__supported.<locals>.one at 0x7fffe89b6820>
E        +  where <function test_set_default_command_when_subparsers_exist__supported.<locals>.two at 0x7fffe8d7fb80> = <bound method ArghNamespace.get_function of ArghNamespace(_functions_stack=[<function test_set_default_command_when_su...0x7fffe89b6820>, <function test_set_default_command_when_subparsers_exist__supported.<locals>.two at 0x7fffe8d7fb80>])>()
E        +    where <bound method ArghNamespace.get_function of ArghNamespace(_functions_stack=[<function test_set_default_command_when_su...0x7fffe89b6820>, <function test_set_default_command_when_subparsers_exist__supported.<locals>.two at 0x7fffe8d7fb80>])> = ArghNamespace(_functions_stack=[<function test_set_default_command_when_subparsers_exist__supported.<locals>.one at 0x7fffe89b6820>, <function test_set_default_command_when_subparsers_exist__supported.<locals>.two at 0x7fffe8d7fb80>]).get_function

test/test_assembling.py:151: AssertionError
_____________________________ test_invalid_choice ______________________________

    def test_invalid_choice():
        def cmd(args):
            return 1

        # root level command

        p = DebugArghParser()
        p.add_commands([cmd])

>       assert run(p, 'bar', exit=True).startswith('invalid choice')
E       assert False
E        +  where False = <built-in method startswith of str object at 0x7fffe89c1c00>('invalid choice')
E        +    where <built-in method startswith of str object at 0x7fffe89c1c00> = "argument {cmd}: invalid choice: 'bar' (choose from 'cmd')".startswith
E        +      where "argument {cmd}: invalid choice: 'bar' (choose from 'cmd')" = run(DebugArghParser(prog='py.test', usage=None, description=None, formatter_class=<class 'argh.constants.CustomFormatter'>, conflict_handler='error', add_help=True), 'bar', exit=True)

test/test_integration.py:380: AssertionError
____________________________ test_explicit_cmd_name ____________________________

    def test_explicit_cmd_name():

        @argh.named('new-name')
        def orig_name():
            return 'ok'

        p = DebugArghParser()
        p.add_commands([orig_name])
>       assert run(p, 'orig-name', exit=True).startswith('invalid choice')
E       assert False
E        +  where False = <built-in method startswith of str object at 0x7fffe89aa530>('invalid choice')
E        +    where <built-in method startswith of str object at 0x7fffe89aa530> = "argument {new-name}: invalid choice: 'orig-name' (choose from 'new-name')".startswith
E        +      where "argument {new-name}: invalid choice: 'orig-name' (choose from 'new-name')" = run(DebugArghParser(prog='py.test', usage=None, description=None, formatter_class=<class 'argh.constants.CustomFormatter'>, conflict_handler='error', add_help=True), 'orig-name', exit=True)

test/test_integration.py:514: AssertionError
=========================== short test summary info ============================
FAILED test/test_assembling.py::test_add_subparsers_when_default_command_exists__supported
FAILED test/test_assembling.py::test_set_default_command_when_subparsers_exist__supported
FAILED test/test_integration.py::test_invalid_choice - assert False
FAILED test/test_integration.py::test_explicit_cmd_name - assert False
============== 4 failed, 71 passed, 2 skipped, 4 xfailed in 0.72s ==============

[stanislavlevin]

test_invalid_choice and test_explicit_cmd_name fail due to python/cpython@0978018.
The expected message would be '%(message)s' -> 'argument %(argument_name)s: %(message)s' (see argparse.ArgumentError).
So, the assertions could be changed to something like assert 'invalid choice' in run(p, 'bar', exit=True)

[stanislavlevin]

test_set_default_command_when_subparsers_exist__supported and test_add_subparsers_when_default_command_exists__supported fail due to python/cpython@a18d522 (Python 3.9.8) which was reverted in python/cpython@587ff7f (Python 3.9.9).