Flake8 3.1.0 to 3.3.0 ignoring errors for radon violations, pre 3.1.0 versions display errors correctly

[asottile]

In GitLab by @robert.peteuil on Apr 28, 2017, 13:40

Environment

flake8 installed via pip, along with pyflakes, pycodestyle, radon and mccabe as seen here:

$flake8 --version
3.3.0 (mccabe: 0.6.1, pycodestyle: 2.0.0, pyflakes: 1.5.0, radon: <property object at 0x109ae1520>) CPython 2.7.10 on Darwin

Problem

• Running flake8 with the --radon-max-cc=1 parameter does not display radon errors
• The -vv results show that it finds them, but ignores them.
• This can be easily reproduced as in the Error Example session.

Error Example

Flake8 3.3.0 results for command flake8 awss/__init__.py --radon-max-cc=1 -v:

• Note: only relevant portion of output included.

flake8.checker            MainProcess    291 INFO     Making checkers
flake8.checker            MainProcess    298 INFO     Checking 1 files
flake8.main.application   MainProcess    502 INFO     Finished running
flake8.main.application   MainProcess    502 INFO     Reporting errors
flake8.main.application   MainProcess    502 INFO     Found a total of 10 violations and reported 0

Flake8 3.3.0 -vv results for the same command:

Each error is in the log, yet does not get reported. In the log, the errors have the suffix of "is not inline ignored"

• Note: only relevant portion of output included.

flake8.main.application   MainProcess   1472 INFO     Finished running
flake8.main.application   MainProcess   1472 INFO     Reporting errors
flake8.style_guide        MainProcess   1473 DEBUG    Deciding if "R701:" should be reported
flake8.style_guide        MainProcess   1473 DEBUG    The user configured "R701:" to be "Ignored.Implicitly", "Selected.Implicitly"
flake8.style_guide        MainProcess   1473 DEBUG    "R701:" will be "Decision.Ignored"
flake8.style_guide        MainProcess   1473 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=147, column_number=1, text="'cmd_list' is too complex (2)", physical_line='def cmd_list(options):\n') is not inline ignored
flake8.style_guide        MainProcess   1473 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=165, column_number=1, text="'cmd_startstop' is too complex (2)", physical_line='def cmd_startstop(options):\n') is not inline ignored
flake8.style_guide        MainProcess   1473 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=194, column_number=1, text="'cmd_ssh' is too complex (3)", physical_line='def cmd_ssh(options):\n') is not inline ignored
flake8.style_guide        MainProcess   1473 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=238, column_number=1, text="'qry_check' is too complex (2)", physical_line='def qry_check(qry_string):\n') is not inline ignored
flake8.style_guide        MainProcess   1473 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=255, column_number=1, text="'qry_create' is too complex (4)", physical_line='def qry_create(options):\n') is not inline ignored
flake8.style_guide        MainProcess   1473 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=305, column_number=1, text="'qry_helper' is too complex (4)", physical_line='def qry_helper(flag_filt, qry_string, title_out, flag_id=False, filt_st=""):\n') is not inline ignored
flake8.style_guide        MainProcess   1473 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=324, column_number=1, text="'list_instances' is too complex (4)", physical_line='def list_instances(title_out, i_info, numbered="no"):\n') is not inline ignored
flake8.style_guide        MainProcess   1474 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=358, column_number=1, text="'det_instance' is too complex (3)", physical_line='def det_instance(command, i_info, title_out):\n') is not inline ignored
flake8.style_guide        MainProcess   1474 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=388, column_number=1, text="'user_picklist' is too complex (2)", physical_line='def user_picklist(title_out, i_info, command):\n') is not inline ignored
flake8.style_guide        MainProcess   1474 DEBUG    Error(code='R701:', filename='awss/__init__.py', line_number=421, column_number=1, text="'user_entry' is too complex (4)", physical_line='def user_entry(entry_raw, command, maxqty):\n') is not inline ignored
flake8.main.application   MainProcess   1474 INFO     Found a total of 10 violations and reported 0'

Expected Results obtained by using Flake8 3.0.4 with -vv:

Reverted to older version & retest:

• pip uninstall flake8
• pip install flake8==3.0.4
• flake8 awss/__init__.py --radon-max-cc=1 -vv

Output using flake8 3.0.4

This displays properly displays the errors.

flake8.main.application   MainProcess    436 INFO     Finished running
flake8.checker            MainProcess    436 INFO     Joining Process-1 to the main process
flake8.checker            MainProcess    436 INFO     Joining Process-2 to the main process
flake8.checker            MainProcess    436 INFO     Joining Process-3 to the main process
flake8.checker            MainProcess    436 INFO     Joining Process-4 to the main process
flake8.checker            MainProcess    436 INFO     Joining Process-5 to the main process
flake8.checker            MainProcess    436 INFO     Joining Process-6 to the main process
flake8.checker            MainProcess    437 INFO     Joining Process-7 to the main process
flake8.checker            MainProcess    437 INFO     Joining Process-8 to the main process
flake8.main.application   MainProcess    437 INFO     Reporting errors
awss/__init__.py:147:1: R701: 'cmd_list' is too complex (2)
awss/__init__.py:165:1: R701: 'cmd_startstop' is too complex (2)
awss/__init__.py:194:1: R701: 'cmd_ssh' is too complex (3)
awss/__init__.py:238:1: R701: 'qry_check' is too complex (2)
awss/__init__.py:255:1: R701: 'qry_create' is too complex (4)
awss/__init__.py:305:1: R701: 'qry_helper' is too complex (4)
awss/__init__.py:324:1: R701: 'list_instances' is too complex (4)
awss/__init__.py:358:1: R701: 'det_instance' is too complex (3)
awss/__init__.py:388:1: R701: 'user_picklist' is too complex (2)
awss/__init__.py:421:1: R701: 'user_entry' is too complex (4)
flake8.main.application   MainProcess    437 INFO     Found a total of 10 violations and reported 10

This is easily reproducible:

• Install 3.3.0, and run flake8 example.py --radon-max-cc=1 -v - no errors displayed
• Uninstall 3.3.0, install 3.0.4, run the same command, errors displayed
• Uninstall 3.0.4, install 3.3.0, no errors displayed

Output of flake8 --bug-report

Here's the flake8 --bug-report output:

Traceback (most recent call last):
  File "/usr/local/bin/flake8", line 11, in <module>
    sys.exit(main())
  File "/Library/Python/2.7/site-packages/flake8/main/cli.py", line 16, in main
    app.run(argv)
  File "/Library/Python/2.7/site-packages/flake8/main/application.py", line 328, in run
    self._run(argv)
  File "/Library/Python/2.7/site-packages/flake8/main/application.py", line 315, in _run
    self.initialize(argv)
  File "/Library/Python/2.7/site-packages/flake8/main/application.py", line 299, in initialize
    self.parse_configuration_and_cli(argv)
  File "/Library/Python/2.7/site-packages/flake8/main/application.py", line 168, in parse_configuration_and_cli
    self.option_manager, argv
  File "/Library/Python/2.7/site-packages/flake8/options/aggregator.py", line 35, in aggregate_options
    original_values, original_args = manager.parse_args(arglist)
  File "/Library/Python/2.7/site-packages/flake8/options/manager.py", line 274, in parse_args
    options, xargs = self.parser.parse_args(args, values)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/optparse.py", line 1400, in parse_args
    stop = self._process_args(largs, rargs, values)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/optparse.py", line 1440, in _process_args
    self._process_long_opt(rargs, values)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/optparse.py", line 1515, in _process_long_opt
    option.process(opt, value, values, self)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/optparse.py", line 789, in process
    self.action, self.dest, opt, value, values, parser)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/optparse.py", line 809, in take_action
    self.callback(self, opt, value, parser, *args, **kwargs)
  File "/Library/Python/2.7/site-packages/flake8/main/debug.py", line 36, in print_information
    print(json.dumps(information(option_manager), indent=2, sort_keys=True))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 250, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 209, in encode
    chunks = list(chunks)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <property object at 0x105f085d0> is not JSON serializable

[asottile]

In GitLab by @robert.peteuil on Apr 28, 2017, 13:41

changed title from Radon Violations (from --radon-ma{-c-}-cc=1) Not Reporting on 3.3.0, but works on old versions, like 3.0.4 (debug and -vv info included) to Radon Violations (from --radon-ma{+x+}-cc=1) Not Reporting on 3.3.0, but works on old versions, like 3.0.4 (debug and -vv info included)

[asottile]

In GitLab by @robert.peteuil on Apr 28, 2017, 14:01

I just noticed that this appears to be related to issue #318, as they both have the suffix "is not inline ignored" appended to items that should be reported and are not.

If this is confirmed, what is the newest version that doesn't have this problem? I need to know which version to specify in my CI scripts.

(UPDATE: as I posted below, 3.0.4 was the last release that doesn't have this issue.)

[asottile]

In GitLab by @robert.peteuil on May 3, 2017, 08:33

changed title from {-Radon Violations (from --radon-max-cc=1) Not Reporting on 3.3.0, but works on old versions, like 3.0.4 (debug and -vv-} info included) to {+Flake8 3.3.0 not displaying errors for radon violations (set with --radon-max-cc=1), works on older versions (-vv and debug+} info included)

[asottile]

In GitLab by @robert.peteuil on May 8, 2017, 11:54

I've done some additional testing:

• flake8-3.0.4 is the last version that properly reports R701 (exceeding --radon-max-cc=n)
• therefore 3.1.0 is likely the release that broke this (unless it's related to a change on the radon side)

Until this is resolved, I've pinned all my CI test scripts to 3.0.4

[asottile]

In GitLab by @robert.peteuil on May 8, 2017, 14:45

changed title from Flake8 3.{-3.0 not displaying errors for radon violations (set with --radon-max-cc=1), works on older versions (-vv and debug info included)-} to Flake8 3.{+1.0 to 3.3.0 ignoring errors for radon violations, pre 3.1.0 versions as expected+}

[asottile]

In GitLab by @robert.peteuil on May 8, 2017, 14:46

changed title from Flake8 3.1.0 to 3.3.0 ignoring errors for radon violations, pre 3.1.0 versions {-as expected-} to Flake8 3.1.0 to 3.3.0 ignoring errors for radon violations, pre 3.1.0 versions {+display errors correctly+}

[asottile]

In GitLab by @sigmavirus24 on May 12, 2017, 16:57

Have you also reported this to Radon (on the chance this is a bug in their code)? Also, your --bug-report output is not very helpful, ostensibly because a plug-in is doing the wrong thing reporting its version information. Does that still happen without Radon installed? I suspect it may have a couple bugs and that we have our own. :)

Thanks for narrowing this down.

Is there a chance you can provide some code where Radon should be displaying an error but doesn't to make this easier for me to test?

[asottile]

In GitLab by @robert.peteuil on May 13, 2017, 13:28

Thanks for following up, Ian.

I have not reported the bug to Radon yet, but i'm open to doing so.
I cannot run this without Radon installed, as the parameter I'm specifying specifically requires Radon.

However, it seemed like a flake8 issue to me for the following reasons:

• All versions of Flake8 are seeing the Radon errors according to the -vv mode output
• flake8 v3.1.0 onwards specifically ignores these errors and appends the suffix "is not inline ignored" in the -vv output
• If Radon is improperly flagging errors - why do they display in flake8 pre 3.1.0?

A two-second overview of radon, I apologize if this is obvious to you:

• Radon calculates the Cyclomatic Complexity of every function in the module or library you point it at
• Radon cannot calculate the CC of a function until it has seen all the lines of the function
• Once a function ends, it calculates the CC
• Since most functions will have a CC greater than 1, the param I used --radon-max-cc=1 should flag nearly every function as a violation

Here some actual code you can test, dump this into a file called example.py:

"""Create CC of 4."""


def main():
    """Create CC of 4."""
    x = 5
    if x:
        pass
    if x % 2:
        pass
    if x > 10:
        pass

You can run radon on it by typing radon cc example.py -s and it will display:

example.py
    F 4:0 main - A (4)

This tell you that, in example.py - function (F), defined at position 4:0, called "main" is graded an "A" and has a CC of (4)

If you run flake8 example.py --radon-max-cc=1 using pre 3.1.0 versions of flake8, it outputs:

example.py:4:1: R701: 'main' is too complex (4)

Note, that in my example - Radon assigns the error to line 4:

• When radon reaches line 13 it realizes that function "main" is complete, and calculates the CC of main
• The calculated CC is compared against the specified max, which was 1
• So an error is flagged at line 4 where function "main" was defined
• Thus when radon is processing line 13 it's assigning an error to line 4 - is this is what's causing the "is not inline" message?

I also noticed:

• running radon directly reports the function starting at 4:0
• flake8 reports it as 4:1
• I suspect this is an unrelated minor display glitch because someone is indexing with zero, and someone else is using 1.

Please let me know if you need anything else.

[asottile]

In GitLab by @sigmavirus24 on May 22, 2017, 18:41

~/s/p/flake8 ❯❯❯ .radon/bin/pip install flake8                                                                                                  master ◼
Collecting flake8
  Using cached flake8-3.3.0-py2.py3-none-any.whl
Collecting enum34; python_version < "3.4" (from flake8)
  Using cached enum34-1.1.6-py2-none-any.whl
Collecting configparser; python_version < "3.2" (from flake8)
Collecting pyflakes<1.6.0,>=1.5.0 (from flake8)
  Using cached pyflakes-1.5.0-py2.py3-none-any.whl
Collecting mccabe<0.7.0,>=0.6.0 (from flake8)
  Using cached mccabe-0.6.1-py2.py3-none-any.whl
Collecting pycodestyle<2.4.0,>=2.0.0 (from flake8)
  Using cached pycodestyle-2.3.1-py2.py3-none-any.whl
Installing collected packages: enum34, configparser, pyflakes, mccabe, pycodestyle, flake8
Successfully installed configparser-3.5.0 enum34-1.1.6 flake8-3.3.0 mccabe-0.6.1 pycodestyle-2.3.1 pyflakes-1.5.0
~/s/p/flake8 ❯❯❯ .radon/bin/flake8 --bug-report                                                                                                 master ◼
{
  "dependencies": [
    {
      "dependency": "setuptools", 
      "version": "35.0.2"
    }
  ], 
  "platform": {
    "python_implementation": "CPython", 
    "python_version": "2.7.13", 
    "system": "Linux"
  }, 
  "plugins": [
    {
      "plugin": "mccabe", 
      "version": "0.6.1"
    }, 
    {
      "plugin": "pycodestyle", 
      "version": "2.3.1"
    }, 
    {
      "plugin": "pyflakes", 
      "version": "1.5.0"
    }
  ], 
  "version": "3.3.0"
}
~/s/p/flake8 ❯❯❯ .radon/bin/pip install radon                                                                                                   master ◼
Collecting radon
  Downloading radon-1.5.0-py2.py3-none-any.whl (43kB)
    100% |████████████████████████████████| 51kB 1.3MB/s 
Collecting mando<0.4,>=0.3 (from radon)
  Downloading mando-0.3.3-py2.py3-none-any.whl
Collecting flake8-polyfill (from radon)
  Using cached flake8_polyfill-1.0.1-py2.py3-none-any.whl
Collecting colorama<0.4,>=0.3 (from radon)
  Downloading colorama-0.3.9-py2.py3-none-any.whl
Requirement already satisfied: flake8 in ./.radon/lib/python2.7/site-packages (from flake8-polyfill->radon)
Requirement already satisfied: enum34; python_version < "3.4" in ./.radon/lib/python2.7/site-packages (from flake8->flake8-polyfill->radon)
Requirement already satisfied: configparser; python_version < "3.2" in ./.radon/lib/python2.7/site-packages (from flake8->flake8-polyfill->radon)
Requirement already satisfied: pyflakes<1.6.0,>=1.5.0 in ./.radon/lib/python2.7/site-packages (from flake8->flake8-polyfill->radon)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in ./.radon/lib/python2.7/site-packages (from flake8->flake8-polyfill->radon)
Requirement already satisfied: pycodestyle<2.4.0,>=2.0.0 in ./.radon/lib/python2.7/site-packages (from flake8->flake8-polyfill->radon)
Installing collected packages: mando, flake8-polyfill, colorama, radon
Successfully installed colorama-0.3.9 flake8-polyfill-1.0.1 mando-0.3.3 radon-1.5.0
~/s/p/flake8 ❯❯❯ .radon/bin/flake8 --bug-report                                                                                                 master ◼
{
  "dependencies": [
    {
      "dependency": "setuptools", 
      "version": "35.0.2"
    }
  ], 
  "platform": {
    "python_implementation": "CPython", 
    "python_version": "2.7.13", 
    "system": "Linux"
  }, 
  "plugins": [
    {
      "plugin": "mccabe", 
      "version": "0.6.1"
    }, 
    {
      "plugin": "pycodestyle", 
      "version": "2.3.1"
    }, 
    {
      "plugin": "pyflakes", 
      "version": "1.5.0"
    }, 
    {
      "plugin": "radon", 
      "version": "1.5.0"
    }
  ], 
  "version": "3.3.0"
}
~/s/p/flake8 ❯❯❯ cat >example.py <<EOF                                                                                                        ⏎ master ◼
> """Create CC of 4."""


def main():
    """Create CC of 4."""
    x = 5
    if x:
        pass
    if x % 2:
        pass
    if x > 10:
        pass
EOF
~/s/p/flake8 ❯❯❯ less example.py                                                                                                                master ◼
"""Create CC of 4."""


def main():
    """Create CC of 4."""
    x = 5
    if x:
        pass
    if x % 2:
        pass
    if x > 10:
        pass
~/s/p/flake8 ❯❯❯ .radon/bin/flake8 --radon-max-cc=1 example.py                                                                                             master ◼
example.py:4:1: R701: 'main' is too complex (4)

So I started to dig into this issue. I'm not sure I can reproduce this.

[asottile]

In GitLab by @sigmavirus24 on May 22, 2017, 18:42

Even if I do .radon/bin/flake8 --isolated --radon-max-cc=1 example.py I still get the appropriate violation. Can you still reproduce this @robertpeteuil ?

[asottile]

In GitLab by @robert.peteuil on May 22, 2017, 20:08

Thank you for taking the time to test this out.

Fortunately, I have somewhat good news for you. As I was re-testing this on my notebook & desktop, I noticed the problem didn't occur outside my project directory.

Therefore, the cause is a config file in my project. I have narrowed down to my tox.ini (where I store my linter settings). It seems odd to me that an potentially improper config had no impact until flake8 3.1.0 and beyond.

Here are the linter rules, verbatim from my tox.ini. Any ideas what is causing this problem? I tried deleting the unnecessary [pycodestyle] section, but that didn't fix it. (the complete file is in the repo).

[flake8]
ignore = D203,H301,H306
select = E,W,F
exclude = .tox,.git,*.pyc,.cache,.eggs,*.egg,build,dist,test.py,zref,__pycache__

[pytest]
python_files = test_*.py
norecursedirs = .**tox** .git .* *.egg* old docs dist build

[doc8]
ignore=D001

[pydocstyle]
ignore = D400

# previously 'pep8'
[pycodestyle]
ignore =

[asottile]

In GitLab by @sigmavirus24 on May 23, 2017, 03:48

Your select list excludes R for the radon checks.

[asottile]

In GitLab by @sigmavirus24 on May 23, 2017, 04:56

(I wrote my last remark from my phone after having just woken up, allow me to expand.)

So Flake8 now supports both ignore and select lists as you're specifying now. With that in mind, we support it so that it can be used in a different way than you're presently using it. I've inferred from your config file that you have the following plugins installed:

• flake8-docstrings
• radon
• hacking

You, however, are telling Flake8 that you only want to see all E* violations, all W* violations, and all F* violations. So all D* violations, all R* violations, and all H* violations are thusly ignored. You can fix this in a couple ways:

[flake8]
ignore = D203,H301,H306
exclude = .tox,.git,*.pyc,.cache,.eggs,*.egg,build,dist,test.py,zref,__pycache__

This will tell Flake8 that anything else is implicitly selected, that will include all non-ignored flake8-docstring violations, all non-ignored hacking violations, and all non-ignored radon violations.

[flake8]
ignore = D203,H301,H306
select = E,W,F,R,D,H
exclude = .tox,.git,*.pyc,.cache,.eggs,*.egg,build,dist,test.py,zref,__pycache__

This will be explicit in telling Flake8 which violation classes you want it to use. That said, if you start using a new plugin, e.g., flake8-import-order then you'll have to update select to also see that output.

Cheers!

[asottile]

In GitLab by @sigmavirus24 on May 23, 2017, 04:56

closed

[asottile]

In GitLab by @robert.peteuil on May 23, 2017, 12:15

Thank you for taking the time to explain it to me in such detail.

This explains why it's not working with 3.1+. But, I'm confused why it worked with prior versions. Why did it display Radon errors prior to version 3.1 with the config as written?

From a user standpoint - It was displaying errors as expected. So, I had no idea that at some arbitrary point I needed to change my configuration file in order for it to continue working.

Can this be included in the flake8 documentation? Adding info about this would have prevented this (I searched all over for a list of items to specify on the select line):

• Perhaps something like this (if true) - "the 'select' line in your configuration file must include all plugins you want reported or they will not be shown in the results."
• A list the Error-Prefixes for the supported plugins (just like you listed them above):
  • D = docstring error
  • R = Radon Error
  • H = Hacking Error
  • ...

Thanks again for the knowledge you shared.