Skip to content

Latest commit

 

History

History
90 lines (55 loc) · 3.85 KB

README.md

File metadata and controls

90 lines (55 loc) · 3.85 KB

Interop Test Runner

The Interop Test Runner automatically generates interoperability matrices by running test cases across different implementations. It currently supports two protocols:

Registered implementations and their Docker images are listed in implementations_quic.json and implementations_webtransport.json.

Live results are published at interop.seemann.io.

Publications

Requirements

The Interop Runner is written in Python 3. You'll need:

  • Python 3 modules:
pip3 install -r requirements.txt

Running the Interop Runner

Run the QUIC interop tests:

python3 run.py

Run WebTransport interop tests:

python3 run.py -p webtransport

Use -s and -c to select specific server and client implementations, and -t to select specific test cases:

python3 run.py -s quic-go -c ngtcp2 -t handshake,transfer

Building an Endpoint

Each implementation is packaged as a Docker image. The test runner communicates with implementations entirely through environment variables and mounted directories.

The test case is passed using the TESTCASE environment variable. If your implementation doesn't support a test case, it MUST exit with status code 127. This allows new test cases to be added without breaking existing implementations.

See quic.md and webtransport.md for protocol-specific setup instructions and test case definitions.

To add your implementation, create a Docker image following the instructions for setting up an endpoint in the quic-network-simulator, publish it on Docker Hub and add it to implementations_quic.json or implementations_webtransport.json. Once your implementation is ready to interop, please send us a PR with this addition.

Multi-Platform Builds

The online interop runner requires linux/amd64 images. If you build on a different architecture (e.g. Apple silicon), use --platform linux/amd64 with docker build.

The recommended approach is a multi-platform build providing both amd64 and arm64 images:

docker buildx create --use
docker buildx build --pull --push --platform linux/amd64,linux/arm64 -t <name:tag> .

IPv6 Support

To enable IPv6 support for the simulator on Linux, the ip6table_filter kernel module needs to be loaded on the host:

sudo modprobe ip6table_filter

Logs

The Interop Runner saves log files to the logs directory (overwritten on each run).

Log files are organized as <server>_<client>/<testcase>/. Each directory contains:

  • output.txt — console output from the test runner (including failure reasons).
  • server/ and client/ — server and client log files.
  • sim/ — pcaps recorded by the simulator.

Implementations that export TLS secrets should use the NSS Key Log format. The SSLKEYLOGFILE environment variable points to a file in the logs directory.

Implementations that support qlog should export log files to the directory specified by the QLOGDIR environment variable.