From 7d124b2ee5b71ae0fc317d5a2aee474ec7ae60d7 Mon Sep 17 00:00:00 2001 From: Bernhard Kaindl Date: Mon, 18 Aug 2025 12:00:00 +0200 Subject: [PATCH] CA-416061: docs: Add how to install and run tests (and replace obsoleted docs) Signed-off-by: Bernhard Kaindl --- CONTRIBUTING.md | 162 +++++++++++++++++++++++++++++++++++------------- README.md | 43 +------------ 2 files changed, 122 insertions(+), 83 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f4092780..673b32c8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,78 +1,153 @@ # Development setup +To run all tests and all supported static analysis tools, Python 3.11 is needed, +which matches the current Python version of XenServer 9. + +Python 3.10 might work as well (when replacing the references in the config files with 3.10). +Python 3.12 and 3.13 can be used too, but not for running [pytype](https://github.com/google/pytype) +([it does not support 3.12 yet](https://google.github.io/pytype/support.html)). + +On Ubuntu, you can install 3.11 (and also 3.12 and 3.13) from the widely-used Python support PPA: + +```sh +sudo add-apt-repository ppa:deadsnakes/ppa && sudo apt update +sudo apt install -y python3.11 python3.12 python3.13 +``` + +If 3.12 or 3.13 are found by [tox](https://tox.wiki), it will run the unit tests with them as well. + +You can also use [uv to install Python versions](https://docs.astral.sh/uv/concepts/python-versions), +see below on a link and an example how to install uv. + +## Do not use distro-privided Python CI tools + +Python tools (other than the Python interpreters themself) provided by Linux distributions +are "always" out of date and do not work as required. If possible, uninstall/remove those, +even if your environment is based on Ubuntu 24.04. In addition, most problematically, the +distribution-provided Python tools are running using the default Pyton version of the +host system, which may not be compatible and can cause subtle errors. (e.g. Python 3.12 +or newer triggers unclear dependency errors in pytype because it is not supported yet) + ## Create a virtual environment with the test dependencies +[Install `uv`](https://docs.astral.sh/uv/), either using `pip`/`pipx` or the +[installer](https://docs.astral.sh/uv/getting-started/installation/) +and install the extras groups that you need. Example: + +```sh +pip install pipx +pipx install uv +uv venv --python 3.11 .uv-venv +. .uv-venv/bin/activate +uv pip install -r pyproject.toml --extra test mypy pytype pyright tox pre-commit +``` + +The older, slower way is to use pip-compile to the deps from `pyproject.toml`: + ```bash python -m venv .venv . .venv/bin/activate -pip install pip-tools==7.3.0 +pip install pre-commit pip-tools==7.3.0 pip-compile --extra=test,mypy,pyright,pytype,tox -o - pyproject.toml | pip install -r /dev/stdin ``` -## Development setup on Fedora 37 +## Running CI -On Fedora 37, the `tox` rpm installs all Python versions. -But this `tox` is older, so install `tox==4.5.1` using `pip` (see below) +These commands assume you installed the tools using the commands above in a Python 3.11 environment. -```bash -sudo dnf install tox;sudo rpm -e tox +### Run pyright, watching for changes and automatically checking the change + +```sh +pyright -w ``` -But preferably use `tox` from the virtual environment instead. +### Run pytest with coverage (fine-grained, e.g. during test development) + +```sh +pytest --cov -v --new-first -x --show-capture=all -rA [optional: files / select which tests to run] +``` -## Development setup on Ubuntu 24.04 +### Watching and running tests on changes automatically using `pytest-watch` (`ptw`) -Prefer the virtual environment. Alternatively, an option is to use `pipx`: +Install ptw in the Python environment using: -```bash -sudo apt install pipx -pipx install tox; pipx install 'pytest<7';pipx install pylint -pipx inject pytest pytest-{forked,localftpserver,pythonpath,subprocess,timeout} pyfakefs pytest_httpserver six mock -pipx inject pylint pyfakefs six mock pytest{,_forked,-localftpserver} +```sh +pip install pytest-watch ``` -Use the `deadsnakes` ppa to install Python versions like 3.8 and 3.11 (see below) +`ptw` watches changed files and runs `pytest` after changes are saved. +Run `ptw`, and pass the files to watch, e.g.: -## Development setup on Ubuntu 22.04 +```sh +ptw tests/test_* +``` -Usage of to install -other python versions. +### Run mypy (fine-grained, e.g. during development) -```bash -sudo apt update -sudo apt install software-properties-common python{2,3}-dev -sudo add-apt-repository -y ppa:deadsnakes/ppa -sudo apt-get install -y python3.{8,11}{,-distutils} +```sh +mypy [optionally pass the flags or files to select which tests to run] +``` + +### Run pylint (fine-grained, e.g. during development) + +```sh +pylint xcp tests [optionally pass the flags or files to select which tests to run] +``` + +### Run all of the above on one go in defined virtual environments + +```sh +tox -e py311-cov-check-lint-mdreport ``` -Installation of additional python versions for testing different versions: +This also checks code coverage and ends with a test report from the pytest run. +If you just run `tox` without arguments, in addition, the unit tests are run with +all installed Python versions (out of the list of 3.11, 3.12 and 3.13) -- If `deadsnakes/ppa` does not work, e.g. for Python 3.6, `conda` or `pyenv` can be used. - For instructions, see : +### Run pre-commit for all checks - ```bash - sudo apt install -y build-essential xz-utils zlib1g-dev \ - lib{bz2,ffi,lzma,readline,ssl,sqlite3}-dev - curl https://pyenv.run | bash # add displayed commands to .bashrc - ~/.pyenv/bin/pyenv install 3.{6,8,11} && ~/.pyenv/bin/pyenv local 3.{6,8,11} # builds them - ``` +To run all tests, including trailing whitespace checks, run -- For testing on newer Ubuntu which has `python2-dev`, but not `pip2`, install `pip2` this way: +```sh +pre-commit run -av +``` - ```bash - curl https://bootstrap.pypa.io/pip/2.7/get-pip.py --output get-pip.py;sudo python2 get-pip.py - ``` +## Alternative: installing pytest packages using `pipx` -You may want to install `pytype` in your user environment to run it directly without `tox`: +`pipx` installs tools in `~/.local/share/pipx/venvs` which can be an alternate +way to install up-to-date python tools ```bash -# On Python != 3.8, pytype can't import xml.dom.minidom, use 3.8: -python3.8 -m pip install pytype -python -m pip install tabulate -./pytype_runner.py +python3.11 -m pip install pipx +pipx install tox; pipx install 'pytest<7';pipx install pylint pyright +pipx inject pytest pytest-{forked,localftpserver,pythonpath,subprocess,timeout} pyfakefs pytest_httpserver six mock +pipx inject pylint pyfakefs six mock pytest{,_forked,-localftpserver} ``` -## Installation of dependencies using `pip` +### Updating the documentation + +For consistently well-spaced documentation, all Markdown files are checked +in CI using Markdownlint, which ensures that e.g. code blocks are separated +by space from the preceeding and following paragraphs and so on. This helps +to keep the Markdown source as well-readable as the rendered Markdown. + +To check and fix Markdown files quickly, use: + +```sh +pre-commit run -av markdownlint +``` + +### Removing trailing whitepace and fixing files to have only one trailing newline + +These fixers detect and fix trailing whitespace and trailing newlines in files +to keep commits clean of adding trailing whitespace and are used in GitHub CI: + +```sh +pre-commit run -av trailing-whitespace +pre-commit run -av end-of-file-fixer +``` + +## Background information on the provided tools ### Testing locally and in GitHub CI using `tox` @@ -156,6 +231,9 @@ For more information to debug `pytest` test suites see ## Running GitHub actions locally using `act` +With `docker` (or `podman`) installed, [act](https://github.com/nektos/act) can be used to run +the CI jobs configured in [.actrc](.actrc): + - `act` uses `docker` (also mimicked by `podman-docker`) to run GitHub actions locally - While `act` does not use the same GitHub runner images, they are similar. diff --git a/README.md b/README.md index b7e5c133..ee12582e 100644 --- a/README.md +++ b/README.md @@ -58,48 +58,9 @@ open a recent workflow run the latest and scroll down until you see the tables! - `.github/act-serial.yaml`: Configuration for the jobs run by the local GitHub actions runner `act` - `.pylintrc`: Configuration file of `Pylint` -## Installation and Setup of a development environment - -For the installation of the general development dependencies, visit [INSTALL.md](INSTALL.md) - -### Updating tests using `pytest-watch` (`ptw`) - -- `pip install pytest-watch` - `ptw` watches changed files and runs `pytest` after changes are saved. - - Then run `ptw` on the code/tests you work on, e.g.: `ptw tests/test_pci_*` and edit the files. - -## Example development workflow - -- Run the tests for at also with `LC_ALL=C python3.6 -m pytest` to check for any `ascii` codec - issues by Python3.6 -- Test with `python2.7 -m pytest` -- Run `mypy` (without any arguments - The configuration is in `pyproject.toml`) -- Run `./pytype_runner.py` -- Run `tox -e py36-lint` and fix any `Pylint` warnings -- Run `tox -e py310-covcombine-check` and fix any missing diff-coverage. -- Run `tox` for the full CI test suite -- Run `act` for the full CI test suite in local containers (similar to GitHub action containers) -- Commit with `--signoff` on a new branch and push it and check the triggered GitHub Action run succeeds. -- Open a new PR - -The list of `virtualenvs` configured in tox can be shown using this command: `tox -av` - -```ml -$ tox -av -default environments: -py36-lint -> Run in a py36 virtualenv: Run pylint and fail on warnings remaining on lines in the diff to master -py311-pyright -> Run in a py311 virtualenv: Run pyright for static analyis -py38-pytype -> Run in a py38 virtualenv: Run pytype for static analyis, intro: https://youtu.be/abvW0mOrDiY -py310-covcombine-check -> Run in a py310 virtualenv: Generate combined coverage reports with py27-test coverage merged Run mypy for static analyis - -additional environments: -cov -> Run in a python virtualenv: Generate coverage html reports (incl. diff-cover) for this environment -covcp -> Run in a python virtualenv: Copy the generated .converage and coverage.xml to the UPLOAD_DIR dir -fox -> Run in a python virtualenv: Generate combined coverage html reports and open them in firefox -mdreport -> Run in a python virtualenv: Make a test report (which is shown in the GitHub Actions Summary Page) -test -> Run in a python virtualenv: Run pytest in this environment with --cov for use in other stages -``` +## Installation and setup of the development environment -If you have only one version of Python3, that works too. Use: `tox -e py-test` +For the installation of the general development dependencies, visit [CONTRIBUTING.md](CONTRIBUTING.md) ## Static analysis using mypy, pylint, pyright and pytype