v0.2.0

rtpbench now tests an agent of your own by its SIP URI, has a third bundled target, speaks SIP
over TLS and SRTP, and can pass or fail a run in CI.

• rtpbench target sip:URI runs the cases, latency and barge-in against any agent from the same
  stack, with digest authentication; transfers go to the stack's operator.
• A third bundled target, Asterisk with AVA, over AudioSocket as upstream ships it or over
  ExternalMedia. AI Voice Connector CE is pinned to upstream main, with its fixes.
• SIP over TLS and SRTP on the caller's leg and through the carrier, judged from the capture with
  the session keys.
• Three cases for SIP messages lost on UDP: invite-retransmit, refer-lost and bye-lost.
• rtpbench check passes or fails a run against a policy, and a GitHub Action runs a target and
  its check on your runner.
• Reports add how evenly the agent's RTP arrives (split from the path's jitter with
  report --agent-pcap), the audio the mock engine received, per-clip barge-in stop rates with
  the backchannels' words, and where the latency and barge-in turns are.
• calibrate measures the wall clock's rate and warns above 100 ppm.
• Every published call's timeline is on the results site.
• Fixes from a clean-room run of the quick start: --network refuses a Docker network that
  would drop the caller's packets (gateway mode), instead of letting every call time out;
  realtime --port for the mock engine on a host where 8000 is taken; a run stopped with
  SIGTERM (systemd, a cancelled CI job) tears its stack down as Ctrl-C does; one release archive
  can be checked and unpacked without the others.
• Every number in the published results now has a command that reproduces it from the archives,
  including the baseline's capacity and held-out tables and the AVA page's counts.

Judging changed: clock steps count only where they are large against the error bar; refer is
inconclusive when the transfer itself failed, and timer when the agent does not use session
timers; barge-in leaves out turns whose speech ended before the interjection; RTP is split by the
capturing side's address. Every published set was judged again with these changes, from its
archive.

Published since 0.1.0, each with its raw runs on a release:
metrology,
media path,
SIP over TLS and SRTP,
Asterisk + AVA,
AI Voice Connector CE at b95f5af
and lost SIP messages.

Needs Linux (or WSL2), Docker 28 or later and uv; start with the
quick start.

Lost SIP messages (rtpbench:fe41c0856bb3): raw runs

Raw run directories behind results/retransmit, as rtpbench wrote them (captures, logs, the mock's recordings), named by the image tag of the code that wrote it. Verify and unpack: sha256sum -c SHA256SUMS && zstd -dc NAME.tar.zst | tar -x.

SIP over TLS and SRTP (rtpbench:f026db2ef9a8): raw runs

Raw run directories behind results/tls-srtp, as rtpbench wrote them (captures, per-call TLS session keys, logs), named by the image tag of the code that wrote it. Verify and unpack: sha256sum -c SHA256SUMS && zstd -dc NAME.tar.zst | tar -x.

Media path (rtpbench:72e41c264960): raw runs

Raw run directories behind results/media-path, as rtpbench wrote them (captures, logs, the mock's recordings), named by the image tag of the code that wrote it. Verify and unpack: sha256sum -c SHA256SUMS && zstd -dc NAME.tar.zst | tar -x.

Asterisk + AVA (rtpbench:0397e3c41635): raw runs

Raw run directories behind results/ava, as rtpbench wrote them (captures, logs, the mock's recordings), named by the image tag of the code that wrote it. Verify and unpack: sha256sum -c SHA256SUMS && zstd -dc NAME.tar.zst | tar -x.

AI Voice Connector CE at b95f5af (rtpbench:f026db2ef9a8): raw runs

Raw run directories behind results/aivc-b95f5af, as rtpbench wrote them (captures, logs, the mock's recordings), named by the image tag of the code that wrote it. Verify and unpack: sha256sum -c SHA256SUMS && zstd -dc NAME.tar.zst | tar -x.

v0.1.0

First public release.

rtpbench calls a voice agent over SIP, captures the RTP on both ends and measures turn latency,
barge-in and SIP behaviour from the packets, on clean and impaired networks (tc netem). Every
timing carries an error bar calibrated against a reference agent, and every case is proven
passing and failing against that agent before it judges yours.

• Calibration of wire timing, speech detection and barge-in stop time, under sourced network
  profiles, with wall-clock steps logged.
• 21 conformance cases (greeting, hold, DTMF, transfer, session timers, codec changes, SIP over
  TCP, and four that only a carrier sends).
• Two bundled targets, LiveKit and AI Voice Connector CE, on a real Realtime engine or a
  deterministic mock, with or without a carrier (OpenSIPS and rtpengine) in front.
• Live progress for long runs; every run records the image tag of the code it ran.

Published: the baseline,
every case on the reference agent and both targets on the mock engine, direct and through the
carrier, 3 network profiles, 10 repeats, with its raw runs on the
baseline release.

Needs Linux (or WSL2), Docker 28 or later and uv; start with the
quick start.

Baseline (rtpbench:ad0a03d9c585): raw runs

Raw run directories behind results/baseline, as rtpbench wrote them (captures, logs, the reference agent's truth), each named by the image tag of the code that wrote it. Verify and unpack: sha256sum -c SHA256SUMS && zstd -dc NAME.tar.zst | tar -x.