Duplicated or even bugged result if launching pylint on the same file multiple time

[kayhayen]

I have a mode for calling pylint on only the changed files from git, and I was giving it the staged changes and the unstages changes, causing sometimes files to duplicate. More than once, this has caused too many branches to be reported on files, where if the file is only given once, this is not happening, and it's a false alarm.

Steps to reproduce

1. using pylint on the same file names, e.g. with code barely below tresholds

Current behavior

False alarms are given

Expected behavior

I think that pylint should recognize, probably through making it absolute paths, if files are really the same, and then only work on one of them, or report on both files separately. Right now it seems as though the source of 2 files gets assigned to one target internally.

pylint --version output

pylint 2.6.0
astroid 2.4.2
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0]

It's happening everywhere as far as I can tell.

Yours,
Kay

[Pierre-Sassoulas]

Hello, thank you for opening the issue, could you provide the real call that is being made when launching pylint that is producing the bug ? Especially thing like the number of core used with -j option and the content of the files.

[kayhayen]

There really is no mystery to it, I need not use any options:

/c/Python38/python -m pylint Mini.py Mini.py 
************* Module Mini
Mini.py:1:0: C0103: Module name "Mini" doesn't conform to snake_case naming style (invalid-name)
Mini.py:1:0: C0114: Missing module docstring (missing-module-docstring)
Mini.py:1:0: E0401: Unable to import 'glf' (import-error)
Mini.py:1:0: W0611: Unused import glf (unused-import)
Mini.py:1:0: C0103: Module name "Mini" doesn't conform to snake_case naming style (invalid-name)
Mini.py:1:0: C0114: Missing module docstring (missing-module-docstring)
Mini.py:1:0: E0401: Unable to import 'glf' (import-error)
Mini.py:1:0: W0611: Unused import glf (unused-import)

----------------------------------------------------------------------
Your code has been rated at -70.00/10 (previous run: -70.00/10, +0.00)

This is on:

$ /c/Python38/python -m pylint --version
pylint 2.6.0
astroid 2.4.2
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)]

What I meant to test is that the import error is reported twice, in fact everything is, that is already annoying, but duplicating errors is not a big deal, however this is:

$ /c/Python38/python -m pylint nuitka/Constants.py nuitka/Constants.py | grep too-many
nuitka\Constants.py:185:0: R0912: Too many branches (22/12) (too-many-branches)
nuitka\Constants.py:233:0: R0912: Too many branches (20/12) (too-many-branches)
nuitka\Constants.py:278:0: R0912: Too many branches (18/12) (too-many-branches)
    # Many many cases to deal with, pylint: disable=too-many-branches,too-many-return-statements
    # pylint: disable=too-many-branches,too-many-return-statements
    # pylint: disable=too-many-return-statements
    # pylint: disable=too-many-return-statements
    # pylint: disable=too-many-return-statements

$ /c/Python38/python -m pylint nuitka/Constants.py | grep too-many

You can find the file I use here: https://github.com/Nuitka/Nuitka/blob/develop/nuitka/Constants.py

Basically it just takes a limit, any limit I assume, that is not exceeded by a factory of 2. I bet that provided the filename many more times will make it worse.

As you can see, it doesn't take any options, although I override default limits, I don't have any pylint config in action in my examples here.

[Pierre-Sassoulas]

Thank you I can reproduce this bug:

With a file a.py being:

# pylint: disable=too-many-return-statements,inconsistent-return-statements,no-else-return,


def getUnhashableConstant(constant):
    constant_type = type(constant)
    if constant_type in (str,):
        return None
    elif constant_type in (dict, list, set):
        return constant
    elif constant_type is tuple:
        for value in constant:
            res = getUnhashableConstant(value)
            if res is not None:
                return res
        return None
    elif constant is Ellipsis:
        return None
    elif constant_type is slice:
        return None
    else:
        assert False, constant_type

pylint a.py:

------------------------------------
Your code has been rated at 10.00/10

pylint a.py a.py a.py a.py a.py a.py a.py a.py a.py a.py

************* Module a
a.py:4:0: R0912: Too many branches (32/26) (too-many-branches)
a.py:4:0: R0912: Too many branches (40/26) (too-many-branches)
a.py:4:0: R0912: Too many branches (48/26) (too-many-branches)
a.py:4:0: R0912: Too many branches (56/26) (too-many-branches)
a.py:4:0: R0912: Too many branches (64/26) (too-many-branches)
a.py:4:0: R0912: Too many branches (72/26) (too-many-branches)
a.py:4:0: R0912: Too many branches (80/26) (too-many-branches)

[Pierre-Sassoulas]

On top of that in 2.15.0-dev0 we also get an additional duplicate-code.

[Pierre-Sassoulas]

The order of files also needs to be considered when fixing this see #3661

[Pierre-Sassoulas]

Closely related to #689

[emcd]

I think that this bug is also related to #6242 .