fix(hrv): refuse SDNN when the R-R stream is banked, not only when it… - #1108
Merged
Conversation
… is over-counted ryanbr#1085 taught the app to refuse SDNN on an over-counted capture. A drain the night after it merged showed that gate is necessary but not sufficient: the 2026-08-06 Oura night measured coverage 1.03, `rrIntegrity=plausible` — no duplication at all, its records tiling the timeline at a fill ratio of 0.990 — and still printed **SDNN 174 ms** against a 40-100 ms physiological range. Over-counting was never what made that number wrong. A BANKED stream stamps a whole record of intervals on one coarse timestamp, so the stored values are a decomposition of a record period rather than beat-to-beat measurements. The per-record SUM is right to ~1%, which is why meanNN and resting HR stay correct and WHOOP-validated, while the individual intervals are not. Coverage cannot see that fault, by construction: it compares beat-time against wall-clock, and a banked night can be textbook. Measured on that night, after the shipped Malik ectopic filter: - within-5-minute SDNN 123 ms, against 30-80 ms physiological; - only 94 ms of the whole-night figure is genuine trend (HR really moves 47-86 bpm); - widening the ectopic window does not reach it — radius 2 -> 20 moves the within-window figure just 124 -> 99 ms. Each interval sits within 20% of its own local median, so no per-beat artifact rule can see this: the fault is in the decomposition, not in outliers. So the gate has to be on the stream's nature, not on a spread statistic: - `beatAccurateFraction(tsSec:rrMs:)` — the fraction of consecutive beats whose wall-clock gap matches their own R-R value. Beat-accurate streams step one interval per beat and measure ~1.0; a banked stream's gaps are 0 s against ~1 s values and it collapses toward 0. - `beatValuesAreTrustworthy(beatAccurateFraction:)` — SDNN is withheld below the boundary. Independent of `beatSpreadIsTrustworthy`; neither implies the other, and both now gate. The boundary is not tuned. The two populations do not overlap near it: a beat-accurate stream measures ~100%, and every banked Oura overnight measured to date sits at **2.6-6.6%** (five nights, 2026-07-29 -> 08-06). RMSSD/pNN50 stay ungated for the same reason they are ungated by the coverage verdict. `unmeasurable` live spot readings are untouched: too-short or mismatched input returns 1.0 and stays trusted, so an honest live capture is never suppressed. The constants duplicate the ones the respiration gate uses for the same judgement (ryanbr#882/ryanbr#883) rather than sharing them — that gate lives in `SleepStager` on a branch that is not upstream. If it lands, the two should collapse onto this definition; the boundary is worth drawing once, in one place, for both. `hrv diag` now carries `beatAccurate=` so the distribution can be gathered from traces that already exist, the same way `coverage=` was added before acting on it. Verification: `swift test` StrandAnalytics 1248, incl. 6 new tests — the decisive one pins a PERFECTLY COVERED banked night (coverage 1.0, verdict plausible) passing the over-count gate and being refused by the new one, which is the case that motivated this. Kotlin twins of each in `HrvRrCoverageTest`. `./gradlew testFullDebugUnitTest` 3,516 tests with the same 3 pre-existing locale failures as clean main (verified on a clean worktree, not assumed). `Strand` (macOS) built locally since `IntelligenceEngine.swift` is app-target Swift that no default CI job compiles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Kyxz22d4v7QHWFCSWTvdJq
ryanbr
pushed a commit
that referenced
this pull request
Aug 8, 2026
…ked (#1127) An Oura night reports a respiratory rate that looks entirely normal while a WHOOP strap worn on the SAME nights measures ~16.4. Respiration is derived by RSA from the R-R stream, and the ring's banked overnight IBI cannot support RSA: it stores R-R VALUES against coarse per-record timestamps, so the beat-to-beat variation RSA reads is not in the data. Fix: respRateFromRR (both platforms) refuses a stream that is not beat-accurate, reusing HRVAnalyzer.beatAccurateFraction / beatValuesAreTrustworthy (#1108) rather than carrying a second copy of the same judgement and its two constants. That is the collapse #1108's PR body promised: one boundary, one definition, so a threshold change moves respiration and SDNN together. WHOOP R-R and the synthetic RSA fixtures measure ~100% and pass unchanged. MEASURED 2026-08-07 on two real nights (31,460 and 30,754 in-bed beats; the real SleepStager.respRateFromRR, plus a validated port to switch the gate off): beatAccurateFraction 0.0246 / 0.0235 (threshold 0.50) shipped path NaN / NaN both protections disabled 13.33 bpm / 13.33 bpm Three things this corrects in the original #883 writeup, all of which strengthen it: 1. The number is 13.33, not the "~7-10" first reported. It sits inside respPlausibleRangeBpm (8-25) and inside the real adult sleeping range, so the range clamp is no protection whatsoever. Only 1 of 113 and 0 of 114 windows fall below 10 bpm. 2. The estimate carries zero information. SHUFFLING or REVERSING the night's R-R values returns the same 13.3333 to four decimals, on both nights. The tachogram's breath band is a flat 1/f shelf with no peak; 13.33 is the peak-picker's own floor on the 4 Hz grid. This is the #194 bar failed outright, which is why the gate is on BANKED-ness and not on whether the output looks sane. 3. The mechanism in the old comment was wrong. The time AXIS is not corrupted: sum(R-R) over wall span is 1.030 / 1.008, so beat-time reconstructs the night to 1-3%. What is unusable is the interval VALUES - the ~6.6 s record decomposed into ~6 intervals whose sum is right to ~1% while the individuals are not beat-to-beat measurements, the same decomposition documented on beatValuesAreTrustworthy. Also documents, and pins with a test, that this gate and #977's splice skip catch OPPOSITE banking geometries and neither subsumes the other: #977 catches banking that TILES time (the real ring - it independently discards 113/113 and 114/114 windows on these nights), this catches banking that COMPRESSES it (the batched fixture, where no gap ever exceeds rsaGapToleranceS and #977 is blind). And one known limitation, pinned deliberately: the gate detects banking by its symptom, and that symptom is repairable. Re-timing each record's beats by cumsum from the record's own timestamp moves the fraction 0.0246/0.0235 -> 0.875/0.863, defeating both this gate and #977 - while the estimate stays 13.3333 and stays unchanged under shuffling. A well-meant decoder change would silently switch respiration back on for a stream carrying no breathing information. testRetiming...knownLimitation fails the day that happens, and says what to do: re-base the gate on provenance, not timestamp shape. Tests: Swift StrandAnalytics 1270/0 (RespRateRsaTests 7/7 incl. 3 new); Android compileFullDebugKotlin clean, 3554 tests / 3 failed - all 3 (AiCoachContextTest, 2x StandardHrSensorFormatTest locale) reproduce on clean upstream/main, pre-existing. No hardware behaviour changes: this is analytics-only, no BLE path touched. Claude-Session: https://claude.ai/code/session_01XXdjctbkxqo359NuuJascK Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
#1085 taught the app to refuse SDNN on an over-counted capture. The drain the night after it merged
showed that gate is necessary but not sufficient: the 2026-08-06 Oura night measured
coverage 1.03,rrIntegrity=plausible— no duplication at all, its records tiling the timeline ata fill ratio of 0.990 — and still printed SDNN 174 ms against a 40-100 ms physiological range.
Over-counting was never what made that number wrong.
A banked stream stamps a whole record of intervals on one coarse timestamp, so the stored values
are a decomposition of a record period rather than beat-to-beat measurements. The per-record SUM is
right to ~1% — which is why
meanNNand resting HR stay correct and WHOOP-validated — while theindividual intervals are not. Coverage cannot see that fault by construction: it compares beat-time
against wall-clock, and a banked night can be textbook.
Evidence
Measured on that night, after the shipped Malik ectopic filter:
And the fault is not reachable by artifact rejection — each interval sits within 20% of its own
local median, so no per-beat rule sees it:
The change
beatAccurateFraction(tsSec:rrMs:)— fraction of consecutive beats whose wall-clock gap matchestheir own R-R value. Beat-accurate streams step one interval per beat and measure ~1.0; a banked
stream's gaps are 0 s against ~1 s values and it collapses toward 0.
beatValuesAreTrustworthy(beatAccurateFraction:)— SDNN withheld below the boundary. Independentof
beatSpreadIsTrustworthy: neither implies the other, and both now gate.hrv diaggainsbeatAccurate=, so the distribution can be gathered from traces that alreadyexist — the same way
coverage=was logged before anything acted on it.The boundary is not tuned. The two populations do not overlap near it: a beat-accurate stream
measures ~100%, every banked Oura overnight measured to date sits at 2.6-6.6% (five nights,
2026-07-29 → 08-06). A mid-range boundary is the honest expression of "anything in between is a
stream we have never seen".
Scope held deliberately narrow
an honest live capture is never suppressed. That is pinned by a test.
rather than sharing them, because that gate lives in
SleepStageron a branch that is notupstream. If it lands, the two should collapse onto this definition — the boundary is worth drawing
once, in one place, for both.
Verification
swift testStrandAnalytics 1,248 tests, incl. 6 new. The decisive one pins a perfectlycovered banked night (coverage 1.0, verdict
plausible) passing the over-count gate and beingrefused by the new one — the exact case that motivated this.
HrvRrCoverageTest../gradlew testFullDebugUnitTest3,516 tests withthe same 3 pre-existing locale failures as clean
main— verified by running those same testson a clean
upstream/mainworktree, not assumed.Strand(macOS) built locally:IntelligenceEngine.swiftis app-target Swift that no default CIjob compiles.
Not claimed
This does not make Oura SDNN correct — it makes the app stop reporting it. Recovering a real HRV
number from this ring needs the ring's own
0x5Dtag (P6d), which is separate work.