Skip to content

Internals Testing

github-actions[bot] edited this page Jul 29, 2026 · 1 revision

Fifty integration-test binaries under tests/, plus unit tests inside src/, plus doctests. The exact total is not repeated here on purpose — it is carried by the homepage stat card and enforced by a pre-commit gate, so any number written on this page would be wrong within a week.

What follows is the map: what each tier asserts, how to regenerate its artifacts, and — most importantly — the roster of gate tests that fail you for reasons that have nothing to do with the code you were writing.

Tiers

Tier Files Asserts
CLI surface cli_test, cli_options_test, cli_help_test, cli_defaults_test, cli_flag_behavior_test Parsing, the default value of every parameter, --help grouping across ~140 flags, and behavior for flags that were once untested.
CLI golden cli_goldens trycmd goldens under tests/cli/ — exact stdout/stderr for a command line.
Integration integration_test, capture_test, rtp_integration_test, pipeline_test, parse_path_test, bootstrap_test, app_servers_test, config_test Capture-to-output pipeline, and the library facades WS2 extracted from main.rs — bootstrap planning is a pure Cli + Config → RunPlan function precisely so it can be tested here. config_test covers the step before planning: config discovery (-f, SIPNAB_CONFIG, --no-config, unknown-key warnings, the missing-file error) driven through the real binary's --dump-config.
TUI tui_snapshot_test, tui_state_test, tui_e2e_test Rendered buffers via ratatui's TestBackend + insta; state-machine transitions; and end-to-end drives of the real binary inside tmux. See TUI testing.
Servers api_test, api_token_test, mcp_stdio_test, mcp_http_test, mcp_token_test, mcp_token_rotation_test, metrics_test, hep_test REST, MCP (stdio and HTTP), signed-token auth and rotation, Prometheus scrapes, HEP ingestion — all end to end against a spawned process.
Security security_test, privilege_drop_test, resource_bounds_test, crash_test Audit regressions, the never-continue-as-root guarantee, attacker-keyed map caps, and crash-report handling with the real binary.
Property & fuzz property_test, smoke_fuzz_test, fuzz_corpus_replay proptest invariants; the always-on stable-toolchain fuzz floor; and — despite its name — a deterministic sweep over an adversarial seed set defined in the file itself, not a replay of fuzz/corpus/ (it opens no files).
Contract json_schema_test, summary_consistency_test, output_behavior_test, error_types_test, api_guidelines_test, wasm_exports_test The shapes other people's code depends on: JSON schemas, cross-surface summary agreement, machine-readable output flags, typed errors, #[non_exhaustive] on growth-prone enums, WASM export list.
Docs enforcement docs_drift_test, dev_docs_drift_test, link_integrity_test, doc_example_coverage_test, config_examples_test, site_journey_test, mockup_alignment_test See the gate roster below.
Governance flag_coverage_test, keybinding_drift_test, feature_gate_test, config_wiring_test Rules about how the project grows, not about a specific behavior.
Meta support_selftest The shared test helpers have their own tests.

tests/support/

Files in a subdirectory of tests/ are not compiled as their own test binaries, so shared helpers are pulled in with an explicit path attribute:

#[path = "support/run.rs"]
mod run;

That idiom is why you will see the same module included by several test files with no mod.rs chain — each test binary compiles its own copy.

Helper Provides
mod.rs Shared normalization (normalize()) used to compare output across platforms; self-tested by support_selftest.
run.rs The canonical binary-spawn helper for CLI/output/config integration tests — one place that knows how to find and invoke the built binary.
server.rs REST API spawn harness: start the server on an ephemeral port, wait for readiness, tear down.
mcp.rs The same for HTTP MCP, including the JSON-RPC framing.
schema.rs JSON-Schema validation against tests/schemas/.
tui_fixtures.rs SIP fixture builders shared by the TUI snapshot and state tests.
fuzz.rs A deterministic xorshift PRNG shared by the stable-toolchain fuzzers, so a failure reproduces from its seed.

Fixtures and corpora

Every command below was run against the current tree and left it unchanged — that is the point of documenting them: if one produces a diff, something has drifted.

Artifact Regenerate with
tests/fixtures/ — synthetic pcaps cargo run --features native --bin gen_fixture
tests/snapshots/ — TUI buffers cargo insta test --features tui --accept (needs cargo install cargo-insta)
tests/cli/ — trycmd goldens TRYCMD=overwrite cargo test --features full --test cli_goldens
tests/schemas/ — JSON Schemas Hand-maintained; json_schema_test validates output against them.
tests/pcap-samples/ — real captures Checked in; never regenerated.
tests/install-sh/ — installer cases Hand-maintained; exercised by the install-sh CI job.
fuzz/corpus/ — fuzz seeds Grown by cargo fuzz run (nightly). Note nothing on the stable toolchain reads this directory: fuzz_corpus_replay drives its own in-file seed set. To make a reproducer run in every cargo test, add it to smoke_fuzz_test as well.

Accepting a snapshot or overwriting a golden is a decision, not a fix. Read the diff first: these files are the record of what the tool promised its users.

The gate-test roster

These fail on changes you did not think were related. Each one exists because the thing it guards silently rotted at least once.

Gate Trips when
docs_drift_test A --flag named in README.md, ../architecture.md or the website does not exist in the CLI; a version marker in the docs or man page disagrees with Cargo.toml; the README feature table misses a Cargo feature; a [theme] slot in ThemeConfig is undocumented, or the slot count quoted in either config reference is wrong. Also the benchmark reproducibility contract: the bench/ harness the benchmarks page tells readers to run must exist and be executable, bench/carrier.py must still produce the corpus composition the page quotes (checked at 1/100 scale), and both doc trees must name the same measured artifact and date.
dev_docs_drift_test A page under docs/internals/ links to a path that no longer exists, names a fn that no longer exists, uses an absolute GitHub URL instead of a relative one, is not registered in build-wiki.py, or breaks a mermaid convention. It also builds the wiki and fails if any relative link survives into the output — the wiki is flat and has no repo tree, so such a link publishes dead. That check runs the generator rather than reading it: the version that only asserted CODE_LINK_RE appeared in the script passed while ](../bench/) was shipping broken.
link_integrity_test Any relative link or heading anchor in either doc tree does not resolve; Zola content uses a plain relative .md link that would render as a dead URL. On the wiki-source side the scan is the top-level docs/*.md plus docs/internals/docs/design/, docs/research/ and docs/superpowers/ are planning material outside the published journey and are deliberately not walked, though a link into them from a scanned page still has to resolve.
doc_example_coverage_test A user-facing CLI flag appears in fewer than two documented examples. A ratchet — the exemption list may only shrink.
flag_coverage_test A new long flag ships with no test referencing it. Also a ratchet: adding a test for a grandfathered flag fails until you remove it from the baseline list.
keybinding_drift_test A key is handled in a controller but not documented in the F1 help.
config_wiring_test A config key exists but is never read, or a CLI flag has no config fallback where its peers do.
feature_gate_test A flag whose subsystem is not compiled in fails late or silently instead of fast and clearly.
api_guidelines_test A growth-prone public enum loses #[non_exhaustive], or a shared store stops being Debug.
summary_consistency_test Two serializing surfaces disagree about a dialog or stream field.
wasm_exports_test The WASM binding loses an export the browser analyzer calls.
site_journey_test / mockup_alignment_test A website journey breaks, or a terminal mockup on the site drifts from real output. Also the homepage's advertised numbers: the binary-size ceiling, the glibc floor, the throughput tiles (which must quote a figure that appears on the benchmarks page they link to), and the two automated-test counts, which must agree with each other and — via quality.yml — with the measured total.
config_examples_test A config sample in the docs no longer parses.
support_selftest The shared normalization helper changes behavior under the tests that depend on it.

The rule for all of them: the gate is not the problem. If flag_coverage_test fails, the flag needs a test; if dev_docs_drift_test fails, a page now lies. Adding an exemption is the last resort, and every exemption list in this repo is written as a ratchet so it cannot quietly grow.

The development loop

Logging. SIPNAB_LOG is a tracing EnvFilter, so it takes levels and per-module directives (SIPNAB_LOG=debug, SIPNAB_LOG=sipnab::rtp=trace). In TUI mode logging is suppressed unless SIPNAB_LOG is set explicitly — the alternative is log lines painting over the interface. -q lowers the default in CLI mode. Configured by init_logging().

Benches. cargo bench --profile profiling. Plain cargo bench cannot build: the cdylib crate type needed for the WASM build forces panic = "abort" into the lib unit while bench units are compiled with unwind, so shared dependencies get built twice with incompatible panic strategies and fail to unify. The profiling profile is release codegen with panic = "unwind" and debug symbols kept, which is also what callgrind wants. Baselines live in benches/BASELINES.md.

nextest. .config/nextest.toml defines three profiles: default (no retries, 30s slow timeout), ci (no retries, no fail-fast, immediate-final output) and e2e (2 retries, 60s timeout) for the timing-sensitive tmux tests. Worth knowing: no workflow currently invokes nextest — CI runs plain cargo test. The config is there for local use and for the day the e2e shim is removed.

The docker lab. harness/ is a docker-compose stack — OpenSIPS, rtpengine, SIPp — for generating real traffic. make up in that directory builds and starts it; make down tears it down.

WASM. Two pre-commit gates cover the browser analyzer: one checks that website/static/wasm/sipnab.js still exports every function the site calls, and one refuses a commit that stages src/wasm.rs without staging a rebuilt bundle alongside it. The rebuild is wasm-pack build --target web --out-dir website/static/wasm --no-typescript -- --no-default-features --features wasm.

Feature matrices. A green cargo test --features full is not proof: CI also builds reduced feature sets, and code behind #[cfg(not(feature = ...))] is invisible to the full build. Before pushing, cargo clippy --all-features --all-targets -- -D warnings and the fuzz workspace check are what the pre-push hook runs for exactly this reason.

Clone this wiki locally