Running Netdev CI tests locally

What is tested by the Netdev CI

This page describes which tests are being executed by the netdev CI (aka NIPA), and how to reproduce them locally.

Important

➡️ It is highly recommended to executed the same tests before any submissions, if you are able to, not to give more work to the maintainers and reviewers

Various tests are executed by the CI. There are currently 3 categories:

• LLM reviews per patch: Sashiko and Clashiko.
• Static tests per patch: builds in different conditions, including the doc, and static analytic checks like checkpatch.pl, Python and shell scripts related ones, etc.
• Runtime tests: executed in VMs: kselftests, KUnit, etc.

LLM reviews

Sashiko can be easily executed locally, with different models. The project's README explains how to install and use it. In short, it is supposed to be easy: configure the model to use, run sashiko init, then sashiko review. That's it.

Clashiko is also using Sashiko, but running with multiple models, doing cross reviews.

Static tests

The netdev CI executes a bunch of scripts from this repo. An easy way to reproduce this is to use the ingest_mdir.py script with your patches:

• export your patches, e.g. using b4 prep/send
• switch to a branch on top of net-next (or net)
• execute ingest_mdir.py

Example:

## Export patches
$ cd $linux
$ b4 prep --set-prefixes 'net-next'  ## or use `git format-patch` with `--subject-prefix="PATCH net-next"`
$ b4 send -o /tmp/my-series          ## or use `git format-patch -o /tmp/my-series/ (...)`

## New branch on top of net-next (or net)
$ git switch -c test <base-branch>  # e.g. net-next or net

## Execute the tests: be patient!
$ cd $nipa
$ ./docker/build/run.sh --pull ./ingest_mdir.py \
        --mdir /tmp/my-series --tree $linux --result-dir out

The last step doesn't have to be executed from a container, but then all dependencies need to be installed manually.

Note

• Images can be built locally using ./docker/build.sh script.
• When using ./docker/build/run.sh, --pull downloads the latest image.
• Docker is being used here, but podman can be used instead.
• Tests can be disabled with ./ingest_mdir.py --disable-test (...) depending on the modified code, e.g. no need to build the kernel when only modifying the selftests. Check ./ingest_mdir.py --list-tests.

KUnit

KUnit instructions can be used here, but in short, you can execute:

$ cd $linux
$ $nipa/docker/selftests/run.sh --pull \
        ./tools/testing/kunit/kunit.py run --alltests --arch=x86_64

Selftests

This part describes how selftests are executed in the netdev CI. It's not the one and only way the tests can be run!

Environment

Virtual tests are executed in Fedora, latest version, with a bunch of dependencies either from the official repos, or using some specific versions built from source. Tests are executed in lightweight VMs with the help of virtme-ng which is used to build the kernel and run the tests.

Tip

The recommended way to execute these tests locally is to use our container images. This can be done by running vng/make/qemu commands from a container started with ${nipa}/docker/selftests/run.sh --pull. Commands can be passed to this script to execute them directly from the container. An alias can be used to have a shorter prefix, see below.

TL;DR

To build and execute all net selftests:

cd $linux
alias run="${nipa}/docker/selftests/run.sh"             ## or the full docker/podman command
docker pull ghcr.io/linux-netdev/nipa-selftests:latest  ## to get the latest version once

TARGETS=(net net/af_unix net/forwarding net/hsr net/mptcp net/netfilter net/openvswitch net/ovpn net/packetdrill net/ppp net/rds net/tcp_ao nci)
for target in "${TARGETS[@]}"; do
    rm -f .config
    run vng --build -v --force --config "tools/testing/selftests/${target}/config"  # --config kernel/configs/debug.config
    run make headers
    run make -C tools/testing/selftests TARGETS="${target}"
    run sudo vng -v --run . --user root -a mitigations=off --rw --cpus 4 --network loop -- \
        make -C tools/testing/selftests TARGETS="${target}" run_tests
done

Feel free to adapt the TARGETS array depending on the modifications you did, e.g. by executing tests from the drivers side as well: drivers/net drivers/net/bonding drivers/net/hw drivers/net/netconsole drivers/net/netdevsim drivers/net/team drivers/net/virtio_net

For more details about the different steps plus some extra tips, please check this dedicated page