Remove pytest-flake8 plugin and call flake8 directly (#2904)

<!--
    Thank you for contributing to our project!

Please do not delete this text completely, but read the text below and
keep
items that seem relevant. If in doubt, just keep everything and add your
    own text at the top, a reviewer will update the checklist for you.

While the checklist is intended to be filled in by the technical and
scientific
reviewers, it is the responsibility of the author of the pull request to
make
    sure all items on it are properly implemented.
-->

## Description

<!--
Please describe your changes here, especially focusing on why this pull
request makes
    ESMValTool better and what problem it solves.

Before you start, please read our contribution guidelines:
https://docs.esmvaltool.org/en/latest/community/

Please fill in the GitHub issue that is closed by this pull request,
e.g. Closes #1903
-->

Sister PR to ESMValGroup/ESMValCore#1722 - quite
needed since the conda-forge package needed some involved pinning of the
flake8 and pytest-flake8 packages. Discussed this and @bouweandela
provided a working solution to the problem of the `pytest_flake8` plugin
being more and more at odds with the mother package `flake8`. Currently
the main `flake8` package (and only one as of this PR) is pinned to <5,
there are still some issues with upper versions but we should have in
mind to unpin and test in the future.

- Closes #2897 
- Link to documentation: No need for it, no change to actual docs

* * *

## Before you get started

<!--
Please discuss your idea with the development team before getting
started,
to avoid disappointment or unnecessary work later. The way to do this is
    to open a new issue on GitHub.
-->

- [x] [☝ Create an
issue](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#contributing-code-and-documentation)
to discuss what you are going to do

## Checklist

It is the responsibility of the author to make sure the pull request is
ready to review. The icons indicate whether the item will be subject to
the [🛠 Technical][1] or [🧪 Scientific][2] review.

<!-- The next two lines turn the 🛠 and 🧪 below into hyperlinks -->
[1]:
https://docs.esmvaltool.org/en/latest/community/review.html#technical-review
[2]:
https://docs.esmvaltool.org/en/latest/community/review.html#scientific-review

- [x] [🛠][1] This pull request has a [descriptive
title](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#pull-request-title)
- [x] [🛠][1] The [list of
authors](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#list-of-authors)
is up to date
- [x] [🛠][1] Any changed dependencies have been [added or removed
correctly](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#dependencies)
- [x] [🛠][1] All [checks below this pull
request](https://docs.esmvaltool.org/en/latest/community/code_documentation.html#pull-request-checks)
were successful

Co-authored-by: Bouwe Andela <b.andela@esciencecenter.nl>

.circleci/config.yml

@@ -26,7 +26,7 @@ commands:
           command: |
             . /opt/conda/etc/profile.d/conda.sh
             conda activate esmvaltool
-            pytest -n 4 -m 'not flake8' --junitxml=test-reports/report.xml
+            pytest -n 4 --junitxml=test-reports/report.xml
             esmvaltool version
             esmvaltool -- --help
             ncl -V
@@ -94,7 +94,7 @@ commands:
           command: |
             . /opt/conda/etc/profile.d/conda.sh
             conda activate esmvaltool
-            pytest -n 4 -m flake8
+            flake8 -j 4
       - test_and_report
       - save_cache:
           key: install-<< parameters.extra >>-{{ .Branch }}-{{ checksum "cache_key.txt" }}
@@ -124,7 +124,7 @@ jobs:
             conda activate esmvaltool
             pip install .[test] > /logs/install.txt 2>&1
             # Run flake8 check with source code (it doesn't work without it)
-            pytest -n 4 -m flake8
+            flake8 -j 4
             # Update/install Julia dependencies
             esmvaltool install Julia > /logs/install_julia.txt 2>&1
             # Remove source to test installed software

.github/workflows/create-condalock-file.yml

@@ -63,6 +63,8 @@ jobs:
         run: esmvaltool --help
       - shell: bash -l {0}
         run: esmvaltool version
+      - shell: bash -l {0}
+        run: flake8
       - shell: bash -l {0}
         run: pytest -n 2 -m "not installation"
       # Automated PR

.github/workflows/install-from-condalock-file.yml

@@ -58,6 +58,8 @@ jobs:
         run: esmvaltool --help
       - shell: bash -l {0}
         run: esmvaltool version 2>&1 | tee source_install_linux_artifacts_python_${{ matrix.python-version }}/version.txt
+      - shell: bash -l {0}
+        run: flake8
       - shell: bash -l {0}
         run: pytest -n 2 -m "not installation"
       - name: Upload artifacts

.github/workflows/test-development.yml

@@ -55,6 +55,9 @@ jobs:
           git clone https://github.com/ESMValGroup/ESMValCore.git
           cd ESMValCore
           pip install -e .[develop]
+      - name: Run flake8
+        shell: bash -l {0}
+        run: flake8
       - name: Run tests
         shell: bash -l {0}
         run: pytest -n 2 -m "not installation" 2>&1 | tee develop_test_linux_artifacts_python_${{ matrix.python-version }}/test_report.txt

.github/workflows/test.yml

@@ -42,6 +42,9 @@ jobs:
       - name: Install Julia dependencies
         shell: bash -l {0}
         run: esmvaltool install Julia
+      - name: Run flake8
+        shell: bash -l {0}
+        run: flake8
       - name: Run tests
         shell: bash -l {0}
         run: pytest -n 2 -m "not installation" 2>&1 | tee test_linux_artifacts_python_${{ matrix.python-version }}/test_report.txt
@@ -83,6 +86,9 @@ jobs:
       - name: Install ESMValTool
         shell: bash -l {0}
         run: pip install -e .[develop] 2>&1 | tee test_osx_artifacts_python_${{ matrix.python-version }}/install.txt
+      - name: Run flake8
+        shell: bash -l {0}
+        run: flake8
       - name: Run tests
         shell: bash -l {0}
         run: pytest -n 2 -m "not installation" 2>&1 | tee test_osx_artifacts_python_${{ matrix.python-version }}/test_report.txt

environment.yml

@@ -63,10 +63,10 @@ dependencies:
   - xgboost>1.6.1  # github.com/ESMValGroup/ESMValTool/issues/2779
   - xlsxwriter
   # Python packages needed for testing
+  - flake8 <5
   - pytest >=3.9,!=6.0.0rc1,!=6.0.0
   - pytest-cov
   - pytest-env
-  - pytest-flake8
   - pytest-html !=2.1.0
   - pytest-metadata >=1.5.1
   - pytest-xdist

setup.cfg

@@ -1,6 +1,5 @@
 [tool:pytest]
 addopts =
-    --flake8
     --doctest-modules
     --ignore=doc/sphinx/source/conf.py
     --cov=esmvaltool
@@ -10,16 +9,18 @@ addopts =
     --numprocesses auto
 env =
     MPLBACKEND = Agg
-flake8-ignore =
-    doc/sphinx/source/conf.py ALL
-    setup.py ALL
 log_level = WARNING
 markers =
     installation: test requires installation of dependencies
 
 [coverage:run]
 parallel = true
 
+[flake8]
+exclude =
+    doc/sphinx/source/conf.py ALL
+    setup.py ALL
+
 [pydocstyle]
 convention = numpy
 

setup.py

@@ -71,11 +71,10 @@
     # Test dependencies
     # Execute `pip install .[test]` once and the use `pytest` to run tests
     'test': [
-        'flake8<4',
+        'flake8<5', # github.com/ESMValGroup/ESMValCore/issues/1696
         'pytest>=3.9,!=6.0.0rc1,!=6.0.0',
         'pytest-cov>=2.10.1',
         'pytest-env',
-        'pytest-flake8>=1.0.6',
         'pytest-html!=2.1.0',
         'pytest-metadata>=1.5.1',
         'pytest-xdist',