Skip to content

fix(hrv): refuse SDNN when the R-R stream is banked, not only when it… - #1108

Merged
ryanbr merged 1 commit into
ryanbr:mainfrom
pipiche38:fix/gate-sdnn-on-banked-rr
Aug 6, 2026
Merged

fix(hrv): refuse SDNN when the R-R stream is banked, not only when it…#1108
ryanbr merged 1 commit into
ryanbr:mainfrom
pipiche38:fix/gate-sdnn-on-banked-rr

Conversation

@pipiche38

Copy link
Copy Markdown

What this fixes

#1085 taught the app to refuse SDNN on an over-counted capture. The drain the night after it merged
showed that gate is necessary but not sufficient: the 2026-08-06 Oura night measured
coverage 1.03, rrIntegrity=plausible — no duplication at all, its records tiling the timeline at
a fill ratio of 0.990 — and still printed SDNN 174 ms against a 40-100 ms physiological range.

Over-counting was never what made that number wrong.

A banked stream stamps a whole record of intervals on one coarse timestamp, so the stored values
are a decomposition of a record period rather than beat-to-beat measurements. The per-record SUM is
right to ~1% — which is why meanNN and resting HR stay correct and WHOOP-validated — while the
individual intervals are not. Coverage cannot see that fault by construction: it compares beat-time
against wall-clock, and a banked night can be textbook.

Evidence

Measured on that night, after the shipped Malik ectopic filter:

measured physiological
whole-night SDNN 174 ms 40-100
within-5-minute SDNN (median, n=106) 123 ms 30-80
of which genuine trend (SD of 5-min medians) 94 ms — HR really moves 47-86 bpm

And the fault is not reachable by artifact rejection — each interval sits within 20% of its own
local median, so no per-beat rule sees it:

ectopic window radius 2 (shipped) 5 12 20
within-5-min SDNN 124 ms 115 105 99
beats kept 91.0% 90.8% 89.9% 89.3%

The change

  • beatAccurateFraction(tsSec:rrMs:) — fraction of consecutive beats whose wall-clock gap matches
    their own R-R value. Beat-accurate streams step one interval per beat and measure ~1.0; a banked
    stream's gaps are 0 s against ~1 s values and it collapses toward 0.
  • beatValuesAreTrustworthy(beatAccurateFraction:) — SDNN withheld below the boundary. Independent
    of beatSpreadIsTrustworthy: neither implies the other, and both now gate.
  • hrv diag gains beatAccurate=, so the distribution can be gathered from traces that already
    exist — the same way coverage= was logged before anything acted on it.

The boundary is not tuned. The two populations do not overlap near it: a beat-accurate stream
measures ~100%, every banked Oura overnight measured to date sits at 2.6-6.6% (five nights,
2026-07-29 → 08-06). A mid-range boundary is the honest expression of "anything in between is a
stream we have never seen".

Scope held deliberately narrow

Verification

  • swift test StrandAnalytics 1,248 tests, incl. 6 new. The decisive one pins a perfectly
    covered banked night
    (coverage 1.0, verdict plausible) passing the over-count gate and being
    refused by the new one — the exact case that motivated this.
  • Kotlin twins of each in HrvRrCoverageTest. ./gradlew testFullDebugUnitTest 3,516 tests with
    the same 3 pre-existing locale failures as clean main — verified by running those same tests
    on a clean upstream/main worktree, not assumed.
  • Strand (macOS) built locally: IntelligenceEngine.swift is app-target Swift that no default CI
    job compiles.

Not claimed

This does not make Oura SDNN correct — it makes the app stop reporting it. Recovering a real HRV
number from this ring needs the ring's own 0x5D tag (P6d), which is separate work.

… is over-counted

ryanbr#1085 taught the app to refuse SDNN on an over-counted capture. A drain the night
after it merged showed that gate is necessary but not sufficient: the 2026-08-06
Oura night measured coverage 1.03, `rrIntegrity=plausible` — no duplication at all,
its records tiling the timeline at a fill ratio of 0.990 — and still printed
**SDNN 174 ms** against a 40-100 ms physiological range.

Over-counting was never what made that number wrong. A BANKED stream stamps a whole
record of intervals on one coarse timestamp, so the stored values are a decomposition
of a record period rather than beat-to-beat measurements. The per-record SUM is right
to ~1%, which is why meanNN and resting HR stay correct and WHOOP-validated, while the
individual intervals are not. Coverage cannot see that fault, by construction: it
compares beat-time against wall-clock, and a banked night can be textbook.

Measured on that night, after the shipped Malik ectopic filter:
  - within-5-minute SDNN 123 ms, against 30-80 ms physiological;
  - only 94 ms of the whole-night figure is genuine trend (HR really moves 47-86 bpm);
  - widening the ectopic window does not reach it — radius 2 -> 20 moves the
    within-window figure just 124 -> 99 ms. Each interval sits within 20% of its own
    local median, so no per-beat artifact rule can see this: the fault is in the
    decomposition, not in outliers.

So the gate has to be on the stream's nature, not on a spread statistic:

  - `beatAccurateFraction(tsSec:rrMs:)` — the fraction of consecutive beats whose
    wall-clock gap matches their own R-R value. Beat-accurate streams step one
    interval per beat and measure ~1.0; a banked stream's gaps are 0 s against ~1 s
    values and it collapses toward 0.
  - `beatValuesAreTrustworthy(beatAccurateFraction:)` — SDNN is withheld below the
    boundary. Independent of `beatSpreadIsTrustworthy`; neither implies the other, and
    both now gate.

The boundary is not tuned. The two populations do not overlap near it: a beat-accurate
stream measures ~100%, and every banked Oura overnight measured to date sits at
**2.6-6.6%** (five nights, 2026-07-29 -> 08-06). RMSSD/pNN50 stay ungated for the same
reason they are ungated by the coverage verdict. `unmeasurable` live spot readings are
untouched: too-short or mismatched input returns 1.0 and stays trusted, so an honest
live capture is never suppressed.

The constants duplicate the ones the respiration gate uses for the same judgement
(ryanbr#882/ryanbr#883) rather than sharing them — that gate lives in `SleepStager` on a branch
that is not upstream. If it lands, the two should collapse onto this definition; the
boundary is worth drawing once, in one place, for both.

`hrv diag` now carries `beatAccurate=` so the distribution can be gathered from traces
that already exist, the same way `coverage=` was added before acting on it.

Verification: `swift test` StrandAnalytics 1248, incl. 6 new tests — the decisive one
pins a PERFECTLY COVERED banked night (coverage 1.0, verdict plausible) passing the
over-count gate and being refused by the new one, which is the case that motivated this.
Kotlin twins of each in `HrvRrCoverageTest`. `./gradlew testFullDebugUnitTest` 3,516
tests with the same 3 pre-existing locale failures as clean main (verified on a clean
worktree, not assumed). `Strand` (macOS) built locally since `IntelligenceEngine.swift`
is app-target Swift that no default CI job compiles.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kyxz22d4v7QHWFCSWTvdJq
@ryanbr
ryanbr merged commit 8757b8f into ryanbr:main Aug 6, 2026
14 checks passed
@pipiche38
pipiche38 deleted the fix/gate-sdnn-on-banked-rr branch August 6, 2026 08:56
ryanbr pushed a commit that referenced this pull request Aug 8, 2026
…ked (#1127)

An Oura night reports a respiratory rate that looks entirely normal while a WHOOP
strap worn on the SAME nights measures ~16.4. Respiration is derived by RSA from
the R-R stream, and the ring's banked overnight IBI cannot support RSA: it stores
R-R VALUES against coarse per-record timestamps, so the beat-to-beat variation RSA
reads is not in the data.

Fix: respRateFromRR (both platforms) refuses a stream that is not beat-accurate,
reusing HRVAnalyzer.beatAccurateFraction / beatValuesAreTrustworthy (#1108) rather
than carrying a second copy of the same judgement and its two constants. That is the
collapse #1108's PR body promised: one boundary, one definition, so a threshold
change moves respiration and SDNN together. WHOOP R-R and the synthetic RSA fixtures
measure ~100% and pass unchanged.

MEASURED 2026-08-07 on two real nights (31,460 and 30,754 in-bed beats; the real
SleepStager.respRateFromRR, plus a validated port to switch the gate off):

  beatAccurateFraction        0.0246 / 0.0235   (threshold 0.50)
  shipped path                NaN / NaN
  both protections disabled   13.33 bpm / 13.33 bpm

Three things this corrects in the original #883 writeup, all of which strengthen it:

1. The number is 13.33, not the "~7-10" first reported. It sits inside
   respPlausibleRangeBpm (8-25) and inside the real adult sleeping range, so the
   range clamp is no protection whatsoever. Only 1 of 113 and 0 of 114 windows fall
   below 10 bpm.

2. The estimate carries zero information. SHUFFLING or REVERSING the night's R-R
   values returns the same 13.3333 to four decimals, on both nights. The tachogram's
   breath band is a flat 1/f shelf with no peak; 13.33 is the peak-picker's own floor
   on the 4 Hz grid. This is the #194 bar failed outright, which is why the gate is on
   BANKED-ness and not on whether the output looks sane.

3. The mechanism in the old comment was wrong. The time AXIS is not corrupted:
   sum(R-R) over wall span is 1.030 / 1.008, so beat-time reconstructs the night to
   1-3%. What is unusable is the interval VALUES - the ~6.6 s record decomposed into
   ~6 intervals whose sum is right to ~1% while the individuals are not beat-to-beat
   measurements, the same decomposition documented on beatValuesAreTrustworthy.

Also documents, and pins with a test, that this gate and #977's splice skip catch
OPPOSITE banking geometries and neither subsumes the other: #977 catches banking that
TILES time (the real ring - it independently discards 113/113 and 114/114 windows on
these nights), this catches banking that COMPRESSES it (the batched fixture, where no
gap ever exceeds rsaGapToleranceS and #977 is blind).

And one known limitation, pinned deliberately: the gate detects banking by its
symptom, and that symptom is repairable. Re-timing each record's beats by cumsum from
the record's own timestamp moves the fraction 0.0246/0.0235 -> 0.875/0.863, defeating
both this gate and #977 - while the estimate stays 13.3333 and stays unchanged under
shuffling. A well-meant decoder change would silently switch respiration back on for a
stream carrying no breathing information. testRetiming...knownLimitation fails the day
that happens, and says what to do: re-base the gate on provenance, not timestamp shape.

Tests: Swift StrandAnalytics 1270/0 (RespRateRsaTests 7/7 incl. 3 new); Android
compileFullDebugKotlin clean, 3554 tests / 3 failed - all 3 (AiCoachContextTest, 2x
StandardHrSensorFormatTest locale) reproduce on clean upstream/main, pre-existing.
No hardware behaviour changes: this is analytics-only, no BLE path touched.


Claude-Session: https://claude.ai/code/session_01XXdjctbkxqo359NuuJascK

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants