Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no-member on argparse result on Python 3.9 (only) #4667

Open
andy-maier opened this issue Jul 4, 2021 · 4 comments
Open

no-member on argparse result on Python 3.9 (only) #4667

andy-maier opened this issue Jul 4, 2021 · 4 comments
Labels
False Positive 🦟 A message is emitted but nothing is wrong with the code Needs reproduction 🔍 Need a way to reproduce it locally on a maintainer's machine

Comments

@andy-maier
Copy link

andy-maier commented Jul 4, 2021

Steps to reproduce

Given a file pylint_no_member.py:

#!/usr/bin/env python

"""
Demonstrates that Pylint raises no-member when accessing existing members
of an argparse result.

Required ingredients:
- This happens only on Python 3.9, but not on earlier Python versions.
- This happens only when argparse.ArgumentParser is subclassed.
- This happens only for the '--logdir' option, but not for the 'name'
  positional argument. However, in our real case with more options, it also
  happened for positional arguments.
"""

import argparse


class SilentArgumentParser(argparse.ArgumentParser):
    """
    argparse.ArgumentParser subclass that silences any errors and exit and
    just raises them as SystemExit.
    """

    def error(self, message=None):
        """Called for usage errors detected by the parser"""
        raise SystemExit(2)

    def exit(self, status=0, message=None):
        """Not sure when this is called"""
        raise SystemExit(status)


def parse_args(argv):
    """
    Parse the command line arguments.
    """
    parser = SilentArgumentParser()

    parser.add_argument('--logdir', type=str, default=None)
    parser.add_argument('name', type=str, default=None)

    parsed_args = parser.parse_args(argv)
    return parsed_args


args = parse_args(['foo'])
print(repr(args))
print(args.name)
print(args.logdir)  # Pylint issues no-member on Python 3.9

Current behavior

On Python 3.9, Pylint raises no-member when accessing the logdir attribute from the argparse option --logdir:

$ pylint --version
pylint 2.9.3
astroid 2.6.2
Python 3.9.1 (default, Feb  1 2021, 20:41:56) 
[Clang 12.0.0 (clang-1200.0.32.29)]

$ pylint -s n -r n pylint_no_member.py 
************* Module pylint_no_member
pylint_no_member.py:49:6: E1101: Instance of 'Namespace' has no 'logdir' member (no-member)

$ ./pylint_no_member.py 
Namespace(logdir=None, name='foo')
foo
None

The same command on Python 3.8 (with the same Pylint version) does not raise the issue:

$ pylint --version
pylint 2.9.3
astroid 2.6.2
Python 3.8.7 (default, Feb  3 2021, 06:31:03) 
[Clang 12.0.0 (clang-1200.0.32.29)]

$ pylint -s n -r n pylint_no_member.py 
# no output

Expected behavior

On Python 3.9, no-member should also not be raised.

@Pierre-Sassoulas Pierre-Sassoulas added the False Positive 🦟 A message is emitted but nothing is wrong with the code label Jul 4, 2021
@DudeNr33
Copy link
Collaborator

DudeNr33 commented Jul 4, 2021

I can reproduce both the false positive on Python 3.9 and that it "works" on Python 3.8.
"Works" is in quotes because it is not like pylint (or better to say, astroid) actually successfully infers that the Namespace object has a member called logdir.
Instead, the TypeChecker.visit_attribute does not emit anything because it skips the check as the node.expr is Uninferable:

@check_messages("no-member", "c-extension-no-member")
    def visit_attribute(self, node):
        """check that the accessed attribute exists

        to avoid too much false positives for now, we'll consider the code as
        correct if a single of the inferred nodes has the accessed attribute.

        function/method, super call and metaclasses are ignored
        """
        if any(
            pattern.match(name)
            for name in (node.attrname, node.as_string())
            for pattern in self._compiled_generated_members
        ):
            return

        try:
            inferred = list(node.expr.infer())
        except astroid.InferenceError:
            return

        # list of (node, nodename) which are missing the attribute
        missingattr = set()

        non_opaque_inference_results = [
            owner
            for owner in inferred
            if owner is not astroid.Uninferable
            and not isinstance(owner, astroid.nodes.Unknown)
        ]
        if (
            len(non_opaque_inference_results) != len(inferred)
            and self.config.ignore_on_opaque_inference
        ):
            # There is an ambiguity in the inference. Since we can't
            # make sure that we won't emit a false positive, we just stop
            # whenever the inference returns an opaque inference object.
            return   # <-- Python 3.8 returns here

With Python 3.9 in contrast, astroid can correctly infer node.expr (giving Const.NoneType and argparse.Namespace), but cannot find the attribute logdir on the Namespace object:

astroid.exceptions.AttributeInferenceError: 'logdir' not found on <ClassDef.Namespace>

@DudeNr33
Copy link
Collaborator

DudeNr33 commented Jul 4, 2021

Interestingly, the reason why args.name does not trigger a no-member message is that node.expr.infer() in this case returns a third result (Uninferable), which leads to the function returning on the same line as it did under Python 3.8.

@bittner
Copy link
Contributor

bittner commented Feb 16, 2022

I have a similar situation with a Python package I maintain. This happens with pylint==2.12.2 for all modern Python versions (3.6.9, 3.7.12, 3.8.12, 3.9.10).

Code Sample

The shell command that returns a Namespace is also blamed by Pylint to have no members. Example:

from cli_test_helpers import shell

result = shell("flake8")

assert result.exit_code == 0

Pylint (incorrectly) complains:

test_example.py:241:15: E1101: Instance of 'Namespace' has no 'exit_code' member (no-member)

Is there something I can adjust in the implementation to make it easier for Pylint to detect the members (and stop complaining for my users), or is this a Pylint shortcoming that can only be fixed inside Pylint?

@DanielNoord
Copy link
Collaborator

You could check if adding generated-members=argparse.Namespace to your config helps. I actually did so myself in a PR for pylint in which we start to use argparse.
It might actually be good to make this a default value.

@Pierre-Sassoulas Pierre-Sassoulas added the Needs reproduction 🔍 Need a way to reproduce it locally on a maintainer's machine label Jul 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
False Positive 🦟 A message is emitted but nothing is wrong with the code Needs reproduction 🔍 Need a way to reproduce it locally on a maintainer's machine
Projects
None yet
Development

No branches or pull requests

5 participants