Skip to content

Score an Oura night by feeding the ring's own hypnogram into the scorer (#804 Fix A) - #877

Closed
pipiche38 wants to merge 1 commit into
ryanbr:mainfrom
pipiche38:fix/oura-hypnogram-into-scorer
Closed

Score an Oura night by feeding the ring's own hypnogram into the scorer (#804 Fix A)#877
pipiche38 wants to merge 1 commit into
ryanbr:mainfrom
pipiche38:fix/oura-hypnogram-into-scorer

Conversation

@pipiche38

@pipiche38 pipiche38 commented Jul 27, 2026

Copy link
Copy Markdown

Closes #804.

An Oura-only night decoded and persisted a complete SleepNet hypnogram (#773) and showed on the Sleep timeline, yet the nightly summary reported it as blank:

sleep day=2026-07-27 totalSleepMin=nil stages=nil eff=nil matched=0 source=computed
hrv day=2026-07-27 window=whole avgHrv=nil

Root cause: the nightly scorer (AnalyticsEngine.analyzeDay) stages sleep from the motion/gravity vector, and a ring streams no gravity at all. So detectSleep returns nothing → the sleep window is empty → avgHrv (computed only over that window) stays nil despite tens of thousands of R‑R intervals present. The ring's persisted hypnogram only ever fed the display‑path SleepMerge, never analyzeDay.

Confirmed on device (app 9.1.1, night 2026‑07‑26→27): the ring produced deep 88 / light 316 / rem 88 / awake 182 min, eff 73%, persisted and won the merge — and the summary was still totalSleepMin=nil / avgHrv=nil. A fully‑formed hypnogram, still scoring blank
— proof the blocker is scorer wiring, not decode.

The fix (general, not Oura‑hardcoded)

analyzeDay gains an optional providedSleep: [SleepSession] = []:

  • When non‑empty, each provided session's nightly restingHR / avgHRV is re‑derived from that day's hr/rr over its window (via the SAME SleepStager.sessionRestingHR / sessionAvgHRV helpers detectSleep uses).
  • Detected sessions that overlap a provided one are dropped (provided is authoritative); a non‑overlapping detected nap survives.
  • Empty providedSleep = byte‑identical motion‑only path.

IntelligenceEngine supplies it: when the day owner sends no usable gravity vector (grav.count < 2) and isn't the canonical WHOOP import namespace, it reads the owner's persisted hypnogram and reconstructs pure SleepSessions from the stored CachedSleepSession.
New shared helpers both platforms: decodeStages + sleepSession(fromProvided:).

Scope / safety

Verification

  • Swift StrandAnalytics 1173/0 (3 new); Kotlin AnalyticsEngineProvidedSleepTest + full analytics suite green; macOS app builds.
  • Tests: a provided hypnogram over a gravity‑free night now scores + derives avgHrv/restingHr; empty providedSleep asserted byte‑identical to omitting (and still nil without gravity — the pre‑fix state).
  • ⏳ Real‑strap validation pending: the next Oura sync should score the 7/26→27 night at ~492 min / eff 73% / non‑nil avgHrv.

Cross‑platform

Swift + Kotlin changed byte‑identically; no schema/migration.

@pipiche38
pipiche38 force-pushed the fix/oura-hypnogram-into-scorer branch from ee7ffa0 to f737d3e Compare July 27, 2026 08:28
@pipiche38

pipiche38 commented Jul 27, 2026

Copy link
Copy Markdown
Author

✅ Real-strap validated (2026-07-27)

Rebuilt NOOP with this change and synced the Oura ring on device. Before Fix A every Oura night was totalSleepMin=nil … avgHrv=nil; now every recent Oura night scores — with sleep stages, efficiency, HRV and RHR.

The night I predicted in the description (7/26→27) came out exactly as expected:

sleep day=2026-07-27 totalSleepMin=492 stages=89+88+316=492 eff=0.73 matched=1 source=computed
hrv   day=2026-07-27 window=whole avgHrv=61.1
hrv diag day=2026-07-27 rmssd=62ms sdnn=186ms meanNN=1007ms coverage=0.98 rrIntegrity=plausible

Scored Oura nights (was all nil before Fix A)

Day Asleep (min) Efficiency avgHrv (ms) RHR floor Integrity
7/27 492 0.73 61.1 50 plausible (cov 0.98)
7/26 366 0.72 75.7 48 plausible
7/25 349 0.65 60.9 51 plausible
7/23 440 0.77 79.7 plausible
7/22 439 0.70 75.0 plausible
7/21 501 0.80 63.6 plausible

All source=computed (Oura era). RMSSD 60–80 ms with meanNN ≈ 1000–1090 ms (~55–60 bpm) is healthy adult overnight HRV — physiologically sane. The scorer is consuming the ring's hypnogram and deriving HRV/RHR from the banked IBI over that window, exactly as intended. Older 7/07–7/20 nights stay imported:whoop with avgHrv=nil (WHOOP import, no ring R‑R — expected, unchanged).

Note (not a regression in this PR)

7/24 reports avgHrv=11.3 but the diagnostic flags it: rr=316/316 coverage=0.00 rrIntegrity=unmeasurable — the ring banked almost no IBI that night (316 beats vs ~35k on good nights), so the derived HRV is meaningless and correctly labelled unmeasurable. This uses the same sessionAvgHRV the WHOOP path already uses, so it isn't new behaviour introduced here — it just surfaces a number where before it was nil. Suppressing avgHrv to nil when rrIntegrity=unmeasurable / coverage ≈ 0 is a worthwhile separate, cross-cutting follow-up (it affects the WHOOP path too), out of scope for this PR.

Dedup(#899) also fired and re-scored the affected days — the existing duplicate-session cleanup working alongside the new scoring, no interaction problem.

Capture d’écran 2026-07-27 à 10 37 06

@vishk23

vishk23 commented Jul 28, 2026

Copy link
Copy Markdown

Sequencing note between this PR and #883 — not an objection, and both are yours, so this is really just a "these two want an order" flag. They touch disjoint files (this one is AnalyticsEngine + IntelligenceEngine, #883 is SleepStager), so nothing in git will enforce it.

Short version: #883 wants to land first.

The reason is a side effect of this PR that isn't obvious from its diff. Verified on main @ e9326000:

analyzeDay derives the nightly respiration by mapping the day's sleep sessions through the RSA estimator —

AnalyticsEngine.swift:580-585
    let perSession = matched
        .map { SleepStager.respRateFromRR(rr, start: $0.start, end: $0.end) }
        .filter { $0.isFinite }

— and today a ring night contributes nothing to matched, because detectSleep bails at SleepStager.swift:909-910 (if grav.count < 2 { return [] }). That's why respRateBpm is nil for a ring night, and your own new test pins it (testEmptyProvidedSleepIsByteIdenticalToOmitting asserts XCTAssertNil(omitted.daily.totalSleepMin)).

This PR makes matched non-empty for exactly that owner. So alongside the sleep and HRV numbers it's meant to unblock, it also — for the first time — routes the ring's banked R-R stream into respRateFromRR. #883's beat-accuracy gate lives inside that function, upstream of everything else in it, so landing #883 first covers this path automatically. The other order leaves a window where the ring's stream is estimated ungated.

Two qualifiers so this doesn't read as scarier than it is: respRateFromRR already clamps to respPlausibleRangeBpm (8.0...25.0, SleepStager.swift:1484 / :1589), which covers the bottom of the range #883 cites; and a spuriously low respiration doesn't trip illness detection, which is elevation-only (IllnessSignalEngine.swift:159-160). It inflates Charge (the recovery resp term is lower-is-better, RecoveryScorer.swift:341-343) and folds into the personal respiration baseline (IntelligenceEngine.swift:818, :845-853, :871). I've written the detail up on #883 rather than duplicating it here.

One scoping observation on this PR specifically, unrelated to the ordering: the gate in IntelligenceEngine is owner != Repository.whoopSource, grav.count < 2, and whoopSource == "my-whoop" (Repository.swift:473; the Kotlin importedDeviceId defaults to the same string, so the twins agree). That excludes only the WHOOP-CSV import namespace — so any gravity-less owner carrying a segment-array stagesJSON qualifies, not just an Oura ring. Apple Health and the Oura cloud import are both in scope. That may well be intended, given you've framed the fix as general rather than Oura-specific, and I think general is the right call. Worth stating explicitly in the PR body though, since "an Oura night" undersells the blast radius and a reviewer sizing the risk would want it.

(On the respiration axis specifically, the Oura cloud path is harmless — it writes no IBI rows, so respRateFromRR dies at its count < 30 guard. It's the BLE ring path that writes R-R.)

Everything above is code-reading on main @ e9326000; I have no Oura ring, so I haven't reproduced any of your device observations. Thanks for splitting this work into focused PRs — it made the interaction between the two much easier to spot. @pipiche38

@pipiche38

Copy link
Copy Markdown
Author

As pointed by @vishk23 , @ryanbr if you are happy to merge the PR, please do merge #883 first and then #877

@pipiche38

Copy link
Copy Markdown
Author

Sequencing note (h/t @vishk23 on #883): merge #883 before this PR.

This PR is what first reaches AnalyticsEngine.swift:651 (respRateFromRR over the main-night window) for an Oura night. Today an Oura-only night has no sleep session (motion staging fails on 0 gravity), so matched is empty and that line never runs — respiration stays nil. Once this PR scores the night from the ring's provided hypnogram, line 651 computes respiration from the banked Oura IBI, which is not beat-accurate → a confidently-wrong ~10 bpm. The existing respPlausibleRangeBpm = 8.0...25.0 clamp does not catch it (~10 is inside the band), so it would be stored and would inflate Rest/recovery and contaminate the respiration baseline.

#883 adds the beat-accuracy precondition inside respRateFromRR (returns NaN → no-data when <50% of beats are time-accurate), which closes that window. So #883 must land first; no change is needed to this PR.

…to the scorer (ryanbr#804 Fix A)

An Oura-only night decoded and persisted a full SleepNet hypnogram (ryanbr#773) and
showed on the Sleep timeline, yet the nightly summary reported
`totalSleepMin=nil eff=nil avgHrv=nil source=computed`. Root cause: the scorer
(`AnalyticsEngine.analyzeDay`) stages sleep from the motion/gravity vector, and a
ring streams NO gravity — so `detectSleep` returned nothing, the sleep window was
empty, and avgHrv (computed only over that window) stayed nil despite tens of
thousands of R-R intervals present. The ring's persisted hypnogram never reached
`analyzeDay`; it only fed the display-path `SleepMerge`.

Fix A (general, not Oura-hardcoded): `analyzeDay` gains an optional `providedSleep`
of already-staged sessions. When the day owner sends no usable gravity vector
(`grav.count < 2` — a ring; a WHOOP always streams gravity) and is not the
canonical WHOOP import namespace, IntelligenceEngine reads the owner's persisted
hypnogram (its own device namespace) and hands it in. Provided sessions bypass
detectSleep + wake refinement; where they overlap a detected session the provided
one wins (authoritative device staging), and a non-overlapping detected nap
survives. Each provided session's nightly restingHR/avgHRV is re-derived from that
day's hr/rr over its window via the SAME helpers detectSleep uses, so RHR/HRV light
up even though the stored ring row carried neither. Empty `providedSleep` (every
WHOOP / pure-function caller) is the byte-identical motion-only path.

Both platforms, byte-identical: `sleepSession(fromProvided:)` / `decodeStages`
reconstruct the pure session from the persisted `CachedSleepSession` (a minute-dict
import row decodes to nothing and is skipped, so only real stage timelines inject).

Tests (no strap): a provided hypnogram over a gravity-free night now yields
totalSleepMin / deep / rem / efficiency AND a derived avgHrv + restingHr; empty
providedSleep is asserted byte-identical to omitting it (and still nil without
gravity — the pre-fix ryanbr#804 state). Swift StrandAnalytics 1173/0; Kotlin
AnalyticsEngineProvidedSleepTest green; macOS app builds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AtXcBU1t6Xk1qJhaQEeDx6
@pipiche38
pipiche38 force-pushed the fix/oura-hypnogram-into-scorer branch from f737d3e to 01ae58e Compare July 30, 2026 07:24
@pipiche38 pipiche38 closed this Jul 31, 2026
@pipiche38

Copy link
Copy Markdown
Author

I'm sorry , but this is not going as my expected speed, and this is an handicap for my development. I don't want to spend too much time in just rebasing all my PRs.
I understand that @ryanbr is the only maintenair. Appreciate the work you are doing.

So I'm closing all those PR .

Feel free to take over what I was doing ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Oura-only night is not scored: the ring's hypnogram never reaches the nightly scorer (motion-based staging can't run on a ring)

2 participants