Basics on Quality Assurance in Python

The basics towards better software with Python by means of development best practices. A work in progress with a continuous improvement mindset.

There are some basic system-wide prerequisites such as python, venv, pip, and git. Next, we will install pipx at user level and use this tool to install pipenv isolated from the general environment. Finally, pyenv is installed to assure that any Python version requested is available and easily switched to (independently of the system Python, it uses a collection of scripts).

pipenv and pre-commit flow diagram.

NOTICE: Using UNIX shell commands in a Debian GNU/Linux Bash shell. Adapt accordingly your Operating System.

Content

• Prerequisites
• Quality Assurance
• Wrap-up
• License

Prerequisites

Linux

macOS

Windows

Quality Assurance

NOTICE: Make sure you've completed the Prerequisites for your operating system case!

Clean and Tidy

/.gitignore

Avoid committing and pushing generated, private, local files. More exclusions may be added at your discretion.

/**/__pycache__/
/.idea/
/build/
/dist/
/*.egg-info/

Code Formatting

isort, black

pipenv install isort black --dev

NOTICE: black and isort may have conflicts, since they both enforce styles in the code (https://pycqa.github.io/isort/docs/configuration/black_compatibility.html). To ensure isort follows the same style as black, add a line in the configuration file as showed below:

pyproject.toml

[tool.isort]
profile = "black"

pipenv run isort .
pipenv run black .

Code Style Enforcement

flake8 + pyproject.toml support = flake8p

pipenv install Flake8-pyproject --dev

pyproject.toml

[tool.flake8]
max-line-length = 120
ignore = ["E203", "E266", "E501", "W503"]
max-complexity = 18
select = ["B", "C", "E", "F", "W", "T4"]

pipenv run flake8p .

Type Checking

mypy

pipenv install mypy --dev

pyproject.toml

[tool.mypy]
files = "."
ignore_missing_imports = true

pipenv run mypy .

Security

bandit, pipenv check

pipenv install bandit[toml] --dev

pyproject.toml

[tool.bandit]
assert_used.skips = "*/tests/*"

pipenv run bandit -c pyproject.toml -r .
pipenv check

Testing

pytest, pytest-cov

pipenv install pytest pytest-cov --dev

pyproject.toml

[tool.pytest.ini_options]
addopts = "--cov --cov-fail-under=100"

[tool.coverage.run]
source = ["."]

[tool.coverage.report]
show_missing = true
omit = ["*/tests/*"]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "raise AssertionError",
    "raise NotImplementedError",
    "if __name__ == .__main__.:"
]

pipenv run pytest

Git Hooks

pre-commit

Putting it all together, i.e., automating while distinguishing Git commit fast-checking requirement from the Git push more time-consuming possible actions such as pytest (including coverage) and pipenv check.

pipenv install pre-commit --dev

.pre-commit-config.yaml

NOTICE: The pipenv check and the pytest (including coverage) are configured to run only on Git push!

repos:
  - repo: local
    hooks:

      ### CODE FORMATTING

      - id: isort
        name: isort
        stages: [ commit ]
        language: system
        entry: pipenv run isort .
        types: [ python ]

      - id: black
        name: black
        stages: [ commit ]
        language: system
        entry: pipenv run black .
        types: [ python ]

      ### CODE STYLE ENFORCEMENT

      - id: flake8
        name: flake8
        stages: [ commit ]
        language: system
        entry: pipenv run flake8p .
        types: [ python ]

      ### TYPE CHECKING

      - id: mypy
        name: mypy
        stages: [ commit ]
        language: system
        entry: pipenv run mypy .
        types: [ python ]
        pass_filenames: false

      ### SECURITY

      - id: bandit
        name: bandit
        stages: [ commit ]
        language: system
        entry: pipenv run bandit -c pyproject.toml -r .
        types: [ python ]

      - id: check
        name: check
        stages: [ push ]
        language: system
        entry: pipenv check
        types: [ python ]

      ### TESTING

      - id: pytest
        name: pytest
        stages: [ push ]
        language: system
        entry: pipenv run pytest
        types: [ python ]
        pass_filenames: false

pipenv run pre-commit install -t pre-commit
pipenv run pre-commit install -t pre-push

It may be run without the Git commit hook triggering. This presents a useful way of testing the .pre-commit-config.yaml calls and also the configuration in pyproject.toml:

pipenv run pre-commit run --all-files --hook-stage commit
pipenv run pre-commit run --all-files --hook-stage push

Wrap-up

All the prerequisites must be accomplished (by following the above instructions or by means of a previous project installation). The project files for quality assurance must be in place (by means of unzipping a download from GitHub).

pipenv install --dev
git init
pipenv run pre-commit install -t pre-commit
pipenv run pre-commit install -t pre-push

Later, you may add your local Git-based repository to a remote, such as, GitLab.

git remote add origin <URL>

License

Copyright 2022 Nuno A. C. Henriques https://nunoachenriques.net

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.