Controlled reproduction: raw-CSI presence detection sits at chance (AUC ≈ 0.58) once time-of-day is controlled — the two apparent successes were confounds

[bcarpio]

First, thanks for open-sourcing this and for being so responsive on the tracker. I'm building toward whole-home presence without any indoor cameras (local Home Assistant, cameras outside only), so WiFi CSI is the one modality that could give me per-room "someone's here" without a lens. I got the hardware up, streaming real CSI, and ran a controlled experiment to completion. Posting as an issue per your steer that reproductions belong here rather than in discussions.

TL;DR: the underlying CSI is real and streams fine at --edge-tier 0, but I could not get reliable boolean presence out of it, and the two results that looked like a breakthrough both collapsed under a same-time-of-day control. This lines up with what others are reporting (#1401 "flickers present in an empty room", #1422 breathing extractor pins to 30 BPM on an empty room). I'd like guidance on whether a phase-aware path or the new ruview-unified model is the intended fix.

Setup

• 3× ESP32-S3-DevKitC-1 (WROOM-1 N16R8), firmware v0.6.7, all re-provisioned --edge-tier 0 (raw CSI passthrough), unique --node-id, --tdm-slot 0/1/2 --tdm-total 3.
• Server ruvnet/wifi-densepose:v0.8.4-esp32, --net=host, --source esp32, /health → "source":"esp32", all three nodes one bedroom, same AP.

Confirmed real hardware, not sim: tcpdump'd the UDP frames from the three ESP32 IPs and hand-decoded against firmware/.../csi_collector.c (ADR-018): magic 0xC5110001, 20-byte LE header, int8 I/Q, 64-subcarrier HT20 (some 128), subcarrier profiles with the expected DC-null shape and real temporal variation, ~35 fps aggregate. The pipeline genuinely works end to end.

Two corrections for others reading this thread

1. Raw CSI is only available at --edge-tier 0. The default (tier 2) does the FFT/DSP on-chip and ships derived scalars, so the recording API shows amplitude: [] and every recorded frame has zero CSI. Configuration gap, not architectural.

2. ruvnet/wifi-densepose-pretrained is not a raw-CSI presence model, and its 82.3% is not presence accuracy. From its own csi-embed-v2.py/config.json it's an 8→64→128 MLP on the 8 pre-extracted summary features, and the 82.3% is heldout_triplet_acc — a contrastive metric on which a random encoder scores 69.6% and raw features 66.4% (csi-embed-v2-metrics.json). Training adds ~13 pts on a representation task; nothing published claims it's a presence classifier.

The experiment — and why the control matters

A naive occupied-vs-empty test is confounded by time-of-day channel drift, so I ran three windows:

• 5.9 h occupied overnight (asleep in the room)
• a fresh empty baseline after leaving
• an occupied-again window 37 min after the empty one — same time of day, so drift is controlled

Amplitude window features, balanced classes, AUC (imbalance-robust), 20–30 draws:

Node	night-occupied vs empty (confounded)	occupied-again vs empty (drift-controlled)
node1	1.000	1.000
node2	0.657	0.607
node3	0.963	0.581
all 3	0.752	0.576

The confounded column looks great (node3 0.96!). The controlled column is the honest one:

• node3's 0.96 was overnight drift — controlled, it drops to 0.581 (chance).
• node1's perfect 1.000 was a link/PHY artifact. During the empty window node1 was on a different PHY: 128 subcarriers, versus 64 subcarriers in both occupied windows — a person does not change your subcarrier count, so the classifier was trivially separating "node1 on HT40" from "node1 on HT20" (a re-association; these nodes grab an AP at boot and occasionally re-associate). I originally also cited an RSSI swing here, but per sensing-server parses CSI-frame RSSI/noise_floor from wrong byte offsets — reports sequence-counter bytes as signal strength #1348 the server's reported RSSI is parsed from the wrong byte offset (sequence-counter bytes, not byte 16), so I'm no longer relying on the dBm figures — the subcarrier-count change, which sensing-server parses CSI-frame RSSI/noise_floor from wrong byte offsets — reports sequence-counter bytes as signal strength #1348 confirms parses correctly, is the reliable tell.
• Controlled, stable-link, combined: AUC ≈ 0.576 — barely above a coin flip with amplitude-only features.

This matches the server's own adaptive classifier for me: after room calibration (recorded train_empty_room/train_still_presence/train_walking/train_active_movement, correctly split into the four classes), training accuracy was 0.45 on 4 classes, and a known-empty room still read "present" 31/31 frames — the same empty-room-reads-present symptom in #1401 and #1422.

Two robustness notes: the per-node AUCs don't depend on cross-node TDM sync, so imperfect TDM isn't the cause; and the same-time-of-day control neutralizes the "fan/AC false-positive" caveat, since a stationary interferer is present in both windows.

Where I think the signal might be — questions

I suspect the info is there and I'm not extracting it. Leading suspects, and I'd love to know which to prioritize:

1. Phase. I used amplitude only and discarded phase — presence/breathing often lives in phase and the 0.1–0.5 Hz band. Is phase the main lever? Recommended phase sanitization (linear detrend / conjugate-multiply between antennas) for these boards?
2. The ruview-unified model. I see the new feat(ruview-unified): unified RF spatial world model work (ADR-273..282, separable delay-Doppler, Gaussian world model). Is that the intended presence path going forward, and is it usable against a tier-0 ESP32-S3 capture today? If so I'll re-run against it.
3. A presence-specific head + the jsonl→rvf path. Is there a presence classifier/head I should load instead of the triplet encoder, and is there a working jsonl → rvf converter now (saw Training does not start – /ws/train/progress returns 404 and no model is generated #1233/Training is badly documented #1322)?
4. Placement / window. Mine were ~4 m apart, one bedroom, not rigorously to the placement guide. How much does correct placement actually move presence AUC in your experience, and would breathing-band spectral features beat my crude window stats?

Offer

I have the 3-node rig up, a clean overnight raw-CSI capture (749k frames, ADR-018 parsed), and Python parse/analysis scripts I'm happy to share so results are comparable. I'll gladly re-run with whatever placement/firmware/feature changes you suggest — or against ruview-unified — and post updated controlled numbers. If a phase-aware model on controlled data is the unlock, I've got idle GPUs to throw at training a room-specific one.

I'd genuinely like to be proven wrong by a phase-aware model on same-time-of-day-controlled captures. My one ask of anyone reporting positive results: run the same empty↔occupied control, because an uncontrolled occupied(night)-vs-empty(morning) comparison will happily hand you an AUC of 0.96 that means nothing.

[ruvnet]

Excellent controlled write-up — this is exactly the kind of rigorous reproduction the project needs.

Short answer to your question: the production path for reliable presence is the per-room calibration pipeline (ADR-151, wifi-densepose-calibration crate), not raw-CSI threshold detection and not ruview-unified.

Why raw CSI sits near chance for presence

Raw amplitude/RSSI metrics without a room-specific baseline are known to be unreliable for boolean presence — environmental reflections, temperature drift, and AP behaviour all shift the baseline between sessions, which is exactly what your time-of-day control caught. The --edge-tier 0 path ships you the I/Q data, but the presence decision layer above it needs a trained per-room model to be meaningful.

The calibration path (ADR-151)

The wifi-densepose-calibration crate implements:

1. Baseline — empty-room Welford amplitude stats + von Mises phase model
2. Enroll — labeled occupancy recording per activity class
3. Extract — feature extraction against the baseline
4. Train — small per-room specialist models (presence, posture, breathing, heartbeat, restlessness, anomaly)

The CLI calibrate and enroll sub-commands drive this. With a per-room baseline and a trained specialist, presence accuracy should be substantially better than what you measured against the out-of-the-box detector.

Regarding ruview-unified (ADR-273)

The accuracy numbers for ruview-unified are currently synthetic (evidence level L0) — no real-data validation has been run yet. It is a research track, not a production fix for the presence problem.

Firmware version note

You're on v0.6.7 firmware; the current release is v0.8.4. Several calibration/presence improvements landed between those versions. Worth upgrading before starting the calibration run.

---

Generated by Claude Code