Repository navigation
Security Panic Audit
Audit record for #140, covering the
unsafe impl Send question and the unwrap()/expect() triage. The sustained-load
and resource-leak portions of that issue are not covered here; they were split
into #154.
- Commit: branch
users/gabloe/remainingm0. - Crates in scope:
crates/felix-wire,crates/felix-broker,crates/felix-transport,crates/felix-storage,services/broker,services/controlplane. - Method: static inventory of
unwrap/expect/panic!/unreachable!/todo!/unimplemented!in non-test code, plus manual review of the binary frame decoders for panics that carry nounwrapat all (slicing,Bufscalar reads, and capacity-driven allocation). - Default (
--no-default-features-equivalent) builds are the baseline. Thetelemetryfeature is opt-in and is called out separately.
Both unsafe impl Send blocks — Broker and EphemeralCache — were removed. Neither
was doing anything: every field of both types is already Send + Sync
(RwLock<HashMap<..>>, atomics, usize, and Box<dyn StorageApi + Send> where
StorageApi: Debug + Send + Sync supplies both auto traits to the trait object). The
compiler's auto-impls already applied, so the unsafe was a no-op that suppressed
nothing and asserted nothing.
Each type now carries a const _ assertion that it is Send + Sync. If a future field
breaks the property, the build fails at the definition rather than surfacing as a
trait-bound error at a distant call site — which is the pressure that produces an
unsafe impl "fix" in the first place.
198 non-test sites total. Classification:
| Class | Count | Disposition |
|---|---|---|
telemetry feature only (timings_telemetry.rs × 3) |
103 | Not in default builds; mutex-poisoning expects in an opt-in diagnostic path |
Test modules in non-tests/ files (tests.rs, bench_ts.rs) |
68 | Test-only |
| Invariant-protected, request path | 14 | Sound; see below |
| Startup / configuration fail-fast | 8 | Intentional: bad config should not produce a half-live process |
| Infallible conversions | 3 | Sound; see below |
| Request-reachable, unguarded | 2 | Fixed; see findings |
-
handlers/publish/control.rs— 12 ×request_id.expect("request id checked"). The protocol invariant is enforced earlier inhandle_publish_message: an acked publish missingrequest_idis rejected withmissing request_id for acked publishbefore any of these sites run. A malformed client frame produces an error response, not a panic. -
handlers/publish/ingress.rs— 2 ×unreachable!("Wait handled above").EnqueuePolicyis config-derived, not request-derived, andWaitis handled in an earlier branch. -
handlers/subscribe/writer.rs—frames.pop().expect("single frame")is guarded by an enclosingframes.len() == 1. -
felix-broker/src/stream_state.rs—next_seq.checked_add(1).expect("log sequence overflow")requires 2^64 publishes to a single stream. -
felix-wire—encode_slice(..).expect("base64 encode slice")writes into a buffer resized to exactlybase64_len(payload.len())?, and that helper returnsErron overflow. This is on the encode path, not decode. -
felix-transport:349—u64::try_from(connection.stable_id())converts fromusize.
decode_publish_batch, decode_event_batch, and decode_shared_event_batch read a
u32 payload count straight off the wire and passed it to Vec::with_capacity before
validating it against the bytes actually present. Per-payload validation inside the loop
was correct, but it ran too late — the allocation had already happened.
Reachability: decode_publish_batch is called from
handle_binary_publish_batch_control before the auth_ctx check, so any peer that
completes a QUIC handshake can reach it. A ~20-byte frame declaring u32::MAX payloads
was sufficient.
Measured impact: a single such frame reserves 95 GiB (Vec<Vec<u8>>) to 127 GiB
(Vec<Bytes>) of address space. Overcommit means one frame typically succeeds, so this
is not a single-shot crash. At roughly a thousand concurrent decodes the address space
is exhausted and the failing allocation calls handle_alloc_error, which aborts —
it does not unwind, so tokio's per-task panic recovery does not contain it. Under a
memory cgroup or strict overcommit, as in a typical container deployment, the threshold
is far lower.
Fix: checked_payload_count rejects any count exceeding remaining / 4, since every
payload requires at least a 4-byte length prefix. Regression tests cover all three
decoders plus a boundary case at the maximum supportable count.
The issue notes a ~560-site raw count. The non-test figure is 198, and 171 of those are
test modules in non-tests/ files or the opt-in telemetry feature. The count on its
own does not indicate risk, and — as F1 shows — the most serious defect in the decode
path involved no unwrap at all. Counting unwrap would not have found it.
- The
telemetryfeature's 103 mutex-poisoningexpects are unreviewed. They are not in default builds; iftelemetryis ever enabled in production, they need their own pass. - Fuzzing the frame and message decoders remains open in
docs/todos.mdand would be the natural way to gain confidence beyond this manual review. - Sustained-load, connection-churn, and resource-leak evidence is not covered by this document; it is tracked in #154. Until that lands, M0's "no known concurrency or leak issues" criterion rests on this static audit alone.
Generated from docs/security/panic-audit.md in the repository. That file is the source of truth; this page may be behind it.
Architecture & Design
- Broker-internal forwarding protocol
- Felix Architecture
- Felix Wire Protocol (v1)
- Felix semantics
- Felix – Comprehensive Design Document
Storage & Replication
- Durable Storage
- Durable Storage Performance
- Felix Durable Segment Format (v2)
- One core log, many semantics
- Plan: group-commit the cache write path
- Shard replication and leader failover
- The cache is a projection over the log
Control Plane & Cluster
- A formal model of one shard
- Control Plane + RAFT Plan (Draft)
- Metadata Raft
- Subscribe routing: redirect, not proxy
- The Postgres the control plane requires
- The local cluster harness
Clients & Configuration
- Auth Design: OIDC Exchange + Felix Tokens
- Broker Config Examples
- Client Config
- Running a client against a cluster
Security
- Bootstrap security
- Panic and abort audit — broker data path
- RBAC Model (Control Plane)
- Soak and resource-leak report — broker
- Threat model: the broker-internal surface
Performance Investigations
- Real-network performance suite
- Sharding investigation: keyed publishes, io_uring, and a ceiling that would not move
- The 0.4.0 performance matrix
- Throughput investigation: why Felix is slower than raw QUIC
Explorations & History