-
Notifications
You must be signed in to change notification settings - Fork 0
Internals Testing
Fifty integration-test binaries under tests/, plus unit tests
inside src/, plus doctests. The exact total is not repeated here on purpose —
it is carried by the homepage stat card and enforced by a pre-commit gate, so
any number written on this page would be wrong within a week.
What follows is the map: what each tier asserts, how to regenerate its artifacts, and — most importantly — the roster of gate tests that fail you for reasons that have nothing to do with the code you were writing.
| Tier | Files | Asserts |
|---|---|---|
| CLI surface |
cli_test, cli_options_test, cli_help_test, cli_defaults_test, cli_flag_behavior_test
|
Parsing, the default value of every parameter, --help grouping across ~140 flags, and behavior for flags that were once untested. |
| CLI golden | cli_goldens |
trycmd goldens under tests/cli/ — exact stdout/stderr for a command line. |
| Integration |
integration_test, capture_test, rtp_integration_test, pipeline_test, parse_path_test, bootstrap_test, app_servers_test, config_test
|
Capture-to-output pipeline, and the library facades WS2 extracted from main.rs — bootstrap planning is a pure Cli + Config → RunPlan function precisely so it can be tested here. config_test covers the step before planning: config discovery (-f, SIPNAB_CONFIG, --no-config, unknown-key warnings, the missing-file error) driven through the real binary's --dump-config. |
| TUI |
tui_snapshot_test, tui_state_test, tui_e2e_test
|
Rendered buffers via ratatui's TestBackend + insta; state-machine transitions; and end-to-end drives of the real binary inside tmux. See TUI testing. |
| Servers |
api_test, api_token_test, mcp_stdio_test, mcp_http_test, mcp_token_test, mcp_token_rotation_test, metrics_test, hep_test
|
REST, MCP (stdio and HTTP), signed-token auth and rotation, Prometheus scrapes, HEP ingestion — all end to end against a spawned process. |
| Security |
security_test, privilege_drop_test, resource_bounds_test, crash_test
|
Audit regressions, the never-continue-as-root guarantee, attacker-keyed map caps, and crash-report handling with the real binary. |
| Property & fuzz |
property_test, smoke_fuzz_test, fuzz_corpus_replay
|
proptest invariants; the always-on stable-toolchain fuzz floor; and — despite its name — a deterministic sweep over an adversarial seed set defined in the file itself, not a replay of fuzz/corpus/ (it opens no files). |
| Contract |
json_schema_test, summary_consistency_test, output_behavior_test, error_types_test, api_guidelines_test, wasm_exports_test
|
The shapes other people's code depends on: JSON schemas, cross-surface summary agreement, machine-readable output flags, typed errors, #[non_exhaustive] on growth-prone enums, WASM export list. |
| Docs enforcement |
docs_drift_test, dev_docs_drift_test, link_integrity_test, doc_example_coverage_test, config_examples_test, site_journey_test, mockup_alignment_test
|
See the gate roster below. |
| Governance |
flag_coverage_test, keybinding_drift_test, feature_gate_test, config_wiring_test
|
Rules about how the project grows, not about a specific behavior. |
| Meta | support_selftest |
The shared test helpers have their own tests. |
Files in a subdirectory of tests/ are not compiled as their own test
binaries, so shared helpers are pulled in with an explicit path attribute:
#[path = "support/run.rs"]
mod run;That idiom is why you will see the same module included by several test files
with no mod.rs chain — each test binary compiles its own copy.
| Helper | Provides |
|---|---|
mod.rs |
Shared normalization (normalize()) used to compare output across platforms; self-tested by support_selftest. |
run.rs |
The canonical binary-spawn helper for CLI/output/config integration tests — one place that knows how to find and invoke the built binary. |
server.rs |
REST API spawn harness: start the server on an ephemeral port, wait for readiness, tear down. |
mcp.rs |
The same for HTTP MCP, including the JSON-RPC framing. |
schema.rs |
JSON-Schema validation against tests/schemas/. |
tui_fixtures.rs |
SIP fixture builders shared by the TUI snapshot and state tests. |
fuzz.rs |
A deterministic xorshift PRNG shared by the stable-toolchain fuzzers, so a failure reproduces from its seed. |
Every command below was run against the current tree and left it unchanged — that is the point of documenting them: if one produces a diff, something has drifted.
| Artifact | Regenerate with |
|---|---|
tests/fixtures/ — synthetic pcaps |
cargo run --features native --bin gen_fixture |
tests/snapshots/ — TUI buffers |
cargo insta test --features tui --accept (needs cargo install cargo-insta) |
tests/cli/ — trycmd goldens |
TRYCMD=overwrite cargo test --features full --test cli_goldens |
tests/schemas/ — JSON Schemas |
Hand-maintained; json_schema_test validates output against them. |
tests/pcap-samples/ — real captures |
Checked in; never regenerated. |
tests/install-sh/ — installer cases |
Hand-maintained; exercised by the install-sh CI job. |
fuzz/corpus/ — fuzz seeds |
Grown by cargo fuzz run (nightly). Note nothing on the stable toolchain reads this directory: fuzz_corpus_replay drives its own in-file seed set. To make a reproducer run in every cargo test, add it to smoke_fuzz_test as well. |
Accepting a snapshot or overwriting a golden is a decision, not a fix. Read the diff first: these files are the record of what the tool promised its users.
These fail on changes you did not think were related. Each one exists because the thing it guards silently rotted at least once.
| Gate | Trips when |
|---|---|
docs_drift_test |
A --flag named in README.md, ../architecture.md or the website does not exist in the CLI; a version marker in the docs or man page disagrees with Cargo.toml; the README feature table misses a Cargo feature; a [theme] slot in ThemeConfig is undocumented, or the slot count quoted in either config reference is wrong. Also the benchmark reproducibility contract: the bench/ harness the benchmarks page tells readers to run must exist and be executable, bench/carrier.py must still produce the corpus composition the page quotes (checked at 1/100 scale), and both doc trees must name the same measured artifact and date. |
dev_docs_drift_test |
A page under docs/internals/ links to a path that no longer exists, names a fn that no longer exists, uses an absolute GitHub URL instead of a relative one, is not registered in build-wiki.py, or breaks a mermaid convention. It also builds the wiki and fails if any relative link survives into the output — the wiki is flat and has no repo tree, so such a link publishes dead. That check runs the generator rather than reading it: the version that only asserted CODE_LINK_RE appeared in the script passed while ](../bench/) was shipping broken. |
link_integrity_test |
Any relative link or heading anchor in either doc tree does not resolve; Zola content uses a plain relative .md link that would render as a dead URL. On the wiki-source side the scan is the top-level docs/*.md plus docs/internals/ — docs/design/, docs/research/ and docs/superpowers/ are planning material outside the published journey and are deliberately not walked, though a link into them from a scanned page still has to resolve. |
doc_example_coverage_test |
A user-facing CLI flag appears in fewer than two documented examples. A ratchet — the exemption list may only shrink. |
flag_coverage_test |
A new long flag ships with no test referencing it. Also a ratchet: adding a test for a grandfathered flag fails until you remove it from the baseline list. |
keybinding_drift_test |
A key is handled in a controller but not documented in the F1 help. |
config_wiring_test |
A config key exists but is never read, or a CLI flag has no config fallback where its peers do. |
feature_gate_test |
A flag whose subsystem is not compiled in fails late or silently instead of fast and clearly. |
api_guidelines_test |
A growth-prone public enum loses #[non_exhaustive], or a shared store stops being Debug. |
summary_consistency_test |
Two serializing surfaces disagree about a dialog or stream field. |
wasm_exports_test |
The WASM binding loses an export the browser analyzer calls. |
site_journey_test / mockup_alignment_test
|
A website journey breaks, or a terminal mockup on the site drifts from real output. Also the homepage's advertised numbers: the binary-size ceiling, the glibc floor, the throughput tiles (which must quote a figure that appears on the benchmarks page they link to), and the two automated-test counts, which must agree with each other and — via quality.yml — with the measured total. |
config_examples_test |
A config sample in the docs no longer parses. |
support_selftest |
The shared normalization helper changes behavior under the tests that depend on it. |
The rule for all of them: the gate is not the problem. If
flag_coverage_test fails, the flag needs a test; if dev_docs_drift_test
fails, a page now lies. Adding an exemption is the last resort, and every
exemption list in this repo is written as a ratchet so it cannot quietly grow.
Logging. SIPNAB_LOG is a tracing EnvFilter, so it takes levels and
per-module directives (SIPNAB_LOG=debug, SIPNAB_LOG=sipnab::rtp=trace).
In TUI mode logging is suppressed unless SIPNAB_LOG is set explicitly — the
alternative is log lines painting over the interface. -q lowers the default
in CLI mode. Configured by init_logging().
Benches. cargo bench --profile profiling. Plain cargo bench cannot
build: the cdylib crate type needed for the WASM build forces
panic = "abort" into the lib unit while bench units are compiled with
unwind, so shared dependencies get built twice with incompatible panic
strategies and fail to unify. The profiling profile is release codegen with
panic = "unwind" and debug symbols kept, which is also what callgrind wants.
Baselines live in benches/BASELINES.md.
nextest. .config/nextest.toml defines three profiles: default (no retries, 30s slow timeout), ci (no retries, no
fail-fast, immediate-final output) and e2e (2 retries, 60s timeout) for the
timing-sensitive tmux tests. Worth knowing: no workflow currently invokes
nextest — CI runs plain cargo test. The config is there for local use and
for the day the e2e shim is removed.
The docker lab. harness/ is a docker-compose stack —
OpenSIPS, rtpengine, SIPp — for generating real traffic. make up in that
directory builds and starts it; make down tears it down.
WASM. Two pre-commit gates cover the browser analyzer: one checks that
website/static/wasm/sipnab.js still exports
every function the site calls, and one refuses a commit that stages
src/wasm.rs without staging a rebuilt bundle alongside it. The rebuild is
wasm-pack build --target web --out-dir website/static/wasm --no-typescript -- --no-default-features --features wasm.
Feature matrices. A green cargo test --features full is not proof: CI
also builds reduced feature sets, and code behind #[cfg(not(feature = ...))]
is invisible to the full build. Before pushing, cargo clippy --all-features --all-targets -- -D warnings and the fuzz workspace check are what the
pre-push hook runs for exactly this reason.
Website · Repository · Issues · Generated from docs/ — edit there, not here.
Getting started
Using the TUI
CLI & automation
Configuration
Integrations (API & MCP)
Development & internals