Code Quality Tooling

[Mischback]

In order to increase the overall code quality, automate the usage of code quality tools (using pre-commit).

This repository contains primarily C code, the actual software for the ESP32. Some utility tools are implemented in different languages, though, e.g. Python.

• C
  • clang-format
  • clang-tidy: Needs compilation database!
  • cppcheck: Needs compilation database!
  • cpplint
  • include-what-you-use: Needs compilation database!
  • Infer: Needs compilation database!
  • oclint: Needs compilation database!
  • lizard
• Python - Most Python-related tooling and the corresponding configuration may be fetched from mischback/django-calingen or mischback/django-consumption. It should be ensured, that these tools (or more specific: the pre-commit hooks) are only applied to actual Python source files.
  • black
  • isort
  • flake8
• CMake
  • find and apply a linter for CMake files (CMakeLists.txt and *.cmake)
  • cmake-lint integrated

[Mischback]

DOES NOT WORK!

All of the following stuff does not work as expected. The referenced pre-commit hooks rely on having the tools available on the actual host system, which pretty much renders them useless as they may be provided as local hooks.

Further investigation reveals nothing similar to this! C / C++ developers seem to handle this in a different way!

Integrate clang-tidy / cppcheck and include-what-you-use

• include this to pre-commit config: https://github.com/Takishima/cmake-pre-commit-hooks
• Test Scenario A
  • fullclean, especially removing the build directory
  • run one of the llvm-based tools THIS IS EXPECTED TO FAIL BADLY!
  • build the project (this also creates the required compilation database)
  • run one of the llvm-based tools: This should work!
• Test Scenario B
  • fullclean
  • build
  • remove content of build directory, except the compilation database (check README of https://github.com/Takishima/cmake-pre-commit-hooks: validation of the build directory is done by a cache file!)
  • run one of the llvm-based tools: This should work!

At this point, we have successfully verified, that the only requirement for running llvm-based tools is the compilation database. This enables us to do the following things:

• create a GitHub Actions workflow, that generates a compilation database (there is a docker-based ESP-IDF action!), upload it as artifact, download the artifact to actually run the linters
• configure the hooks to look for the build directory in a specific place for CI (e.g. ci_build), this is where the compilation database may be provided (see documentation of https://github.com/Takishima/cmake-pre-commit-hooks)
• enable the hooks to (re-) generate the compilation database locally; this might require the use of some advanced pre-commit configuration, as the cmake command must be run with specific environment variables (in order to pick up all the ESP-IDF magic)

[Mischback]

Does not work!

Ok, this did not work! Making clang/llvm-based tools work with ESP-IDF is a much bigger effort than I like to invest at this point.

However, my development system (Debian Bullseye) has a very outdated version of cppcheck, that does work without the llvm-based tooling in place, so at least that one is included.

Integrate include-what-you-use locally

• manually install include-what-you-use, e.g. apt-get install include-what-you-use
  • track the required installation overhead: x kB
• perform a full build and track the required time: x seconds
• Method A: Integrate include-what-you-use in the build process:
  • Reference 1: https://www.kitware.com/static-checks-with-cmake-cdash-iwyu-clang-tidy-lwyu-cpplint-and-cppcheck/
  • Reference 2: https://ortogonal.github.io/cmake-clang-tidy/
  • Reference 3: https://www.danielsieger.com/blog/2021/12/21/clang-tidy-cmake.html
  • modify the project's main CMakeLists.txt (as in Ref 2/3) or the actual build command (as described in Ref 1)
  • perform a full build and track the required time: x seconds
  • Which files are evaluated? Which are reported? We only care about our own codebase!
• Method B: Integrate include-what-you-use in the commit process:
  • add pre-commit hook from https://github.com/Takishima/cmake-pre-commit-hooks
  • make it work after a successful build = compilation database already available
  • make it work without a successful build = compilation database has to be created
    • This probably needs heavy adjustment, so that the hook triggers cmake with all of ESP-IDF's magic (PATH or ENV adjustments?)

Intergrate in CI

Both methods are working locally, but Method A will be difficult in CI, because the whole ESP-IDF toolchain would have to be put in place.

• Build a Github Action workflow
  • include building (https://github.com/espressif/esp-idf-ci-action) as the first step and fetch the required artifacts to make the linters work (Github upload-artifact action?)
  • Method A: System installation
    • keep track of all required linters in a dedicated file (e.g. requirements/apt/linters.txt)
    • install these packages using apt as part of the workflow
    • PROBLEM Linux runner is ubuntu, development machine is debian, so the packages may be out of sync (and may be means will be)
    • download the compilation database (Github download-artifact action?) to the correct path
    • run the linters via pre-commit, using dedicated make recipes (see mischback/django-calingen)
  • Method B: Docker container
    • keep track of all required linters in a dedicated file (e.g. requirements/apt/linters.txt)
    • build a dedicated Dockerfile, based on debian to provide identical versions of the linters
    • PROBLEM Additional complexity because of Docker
    • download the compilation database (Github download-artifact action?) to the correct path
    • PROBLEM How to run the linters? (Reference: github/super-linter)

[Mischback]

Installation Sizes

• include-what-you-use
  • Debian package name: iwyu
  • Bullseye version: 0.15
  • Installation Size: 277 MB
• clang-tidy
  • Debian package name: clang-tidy
  • Bullseye version: 11.0.1
  • Installation Size: 298 MB
• cppcheck
  • Debian package name: cppcheck
  • Bullseye version: 2.3
  • Installation Size: 11 MB

Combined

apt-get install iwyu clang-tidy cppcheck

315 MB