Skip to content

feat(sleep): motion-aware wake refinement behind an Experimental toggle (#364 follow-up) - #402

Merged
ryanbr merged 1 commit into
ryanbr:mainfrom
vishk23:motion-aware-wake
Jul 14, 2026
Merged

feat(sleep): motion-aware wake refinement behind an Experimental toggle (#364 follow-up)#402
ryanbr merged 1 commit into
ryanbr:mainfrom
vishk23:motion-aware-wake

Conversation

@vishk23

@vishk23 vishk23 commented Jul 13, 2026

Copy link
Copy Markdown

What

Follow-up to #364 "Proposal 2 (motion-aware wake classification) — great, but a follow-up", and a companion to the sparse-motion discipline from #345.

Adds WakeMotionRefinement (Swift Packages/StrandAnalytics + a Kotlin twin in com.noop.analytics): a post-pass over the already-staged hypnogram (V1 or V2, unmodified) that reclassifies a scored wake segment to light when its per-minute step-tick cadence shows no locomotion and its per-minute gravity posture is stable outside a minority of isolated turn-over burst minutes (which are kept as wake, ±1 min). It never touches the stagers' own HR-led emission models — both call "wake" primarily off HR/HR-variability, and this pass is exactly the locomotion/posture cross-check that path skips.

The real-world failure this fixes

A WHOOP 5.0 night with pharmacologically-elevated heart rate (a supplement protocol, not illness) scored 194 min wake across ~6.4h in bed (efficiency 0.50), though the wearer reported sleeping through. Minute-level review of the four largest wake blocks found zero walking cadence: each was a single-minute turn-over burst (20-36 step-motion-counter ticks, a posture-variance spike) bracketed by minutes of complete stillness (posture variance < 0.01, zero ticks) — recurring on a ~90-min rhythm consistent with hot-but-atonic REM being scored as wake, not a real awakening. The phone recorded zero steps all night. A genuine get-up looks entirely different: sustained multi-minute walking cadence, not an isolated single-minute blip. No personal data is included here or in the diff — every test fixture is synthetic.

Density self-gate (#345), not a family string

ryanbr flagged in #364 that WHOOP 4.0's gravity stream is often too sparse to tell "restless in bed" from "out of bed," so motion-aware classification would need to self-limit to dense-motion straps. This lands that as a genuine data gate, not a model check: isMotionDense requires that at least 80% of the scored window's minutes carry enough gravity samples to compute a real per-minute variance (a single sample has zero variance by construction and would silently rubber-stamp "stable") and enough step records to know a zero-tick minute is a real zero rather than a missing stream. A WHOOP 4.0 fails this on the data itself every time — it never emits a step sample at all, and its gravity is banked at roughly one vector per minute — so it's untouched regardless of the toggle. A WHOOP 5.0/MG streams both channels densely and is this refinement's expected beneficiary. Threshold numbers are open to tuning; they're fixed a-priori from the one reference night above, not fit to labels, exactly like the honesty bar the sparse-gravity work set.

Honesty / default-off

Ships behind PuffinExperiment.motionAwareWake (Strand/BLE/PuffinExperiment.swift + the Kotlin twin), default OFF — a derived physiological signal with one reference night behind it, per the repo's rule for exactly this situation. The pass only ever shrinks an already-scored wake segment; it never invents wake time, and detection plus the V1/V2 staging engines are byte-identical either way. Toggle row added next to "Sleep staging (V2)" in Settings → Experimental on both macOS/iOS and Android.

Tests (synthetic fixtures only — no real user data)

  • testHotButStillNightReclaimsTheStillSpansAroundIsolatedBursts — a still night with a turn-over burst every 90 min: the still spans reclaim to light, the two burst blocks (±1 min) stay wake, total duration is exactly conserved.
  • testRealGetUpStaysWake — 3 consecutive minutes at 25 ticks/min (a real get-up) trips the sustained-walk locomotion gate; the segment is left untouched.
  • testSparseMotionNightDeclinesToAct — the same burst pattern as the first test, but WHOOP-4.0-shaped density (sparse gravity, no step samples): the gate declines and output is unchanged, isolating density as the only variable.
  • testToggleOffIsByteIdenticalPassthroughenabled=false is byte-identical on a fixture proven live when enabled (so the assertion isn't vacuous).
  • testRecoversDifferentInjectedReclaimAmountsAcrossPatterns — the repo's hard rule for a derived signal ("prove the method tracks a varying input, not one lucky match"): five distinct synthetic patterns (single burst, well-separated bursts, adjacent bursts whose padding overlaps, edge-clamped bursts) each inject a different reclaim amount, and each is recovered exactly.
  • testSessionLevelRefineRecomputesEfficiency — the SleepSession/DetectedSleep convenience wrapper recomputes efficiency from the refined stages and carries restingHR/avgHRV over unchanged.

Mirrored 1:1 in WakeMotionRefinementTest.kt.

Verification:

  • swift test in Packages/StrandAnalytics: 1026/1026 passing (full suite + 6 new).
  • xcodegen generate && xcodebuild -scheme Strand -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO build (the Settings toggle row touches app-target UI): BUILD SUCCEEDED.
  • ./gradlew compileFullDebugKotlin: clean.
  • ./gradlew testFullDebugUnitTest: 2632/2632 passing (full app suite + 6 new), 1035/1035 in com.noop.analytics alone.
  • ./gradlew assembleFullDebug: BUILD SUCCESSFUL.

Android

The Kotlin analytics layer structurally mirrors Swift closely enough to transcribe directly, not just conceptually: StageSegment/DetectedSleep are field-for-field twins of StageSegment/SleepSession, SleepStageHealer.restageFromSamples is the exact analog of Repository.restageFromRaw, and PuffinExperiment.kt already mirrors the Swift toggle registry key-for-key. So this PR transcribes the full post-pass (WakeMotionRefinement.kt, same constants, same algorithm) rather than skipping it, and wires it through the same call sites the existing experimentalSleepV2 flag uses end to end: SleepStageHealer, both IntelligenceEngine.analyzeRecent/analyzeRecentOnCpu signatures, the three Context-aware callers (AppViewModel.kt x2, WhoopBleClient.kt), and a SettingsScreen.kt toggle row next to the existing V2 switch.

Threshold tuning

The constants (sustainedWalkTicksPerMinute=10, singleMinuteWalkTicks=40, stablePostureVarianceG2=0.05, minStableMinuteFraction=0.80, burstPadMinutes=1, the density-gate floors) are named, documented, and fixed a-priori from the one reference night in the issue thread — happy to tune any of them against more real 5.0/MG data if you have nights to compare against, or to adjust the density-gate floors if they're stricter/looser than the sparse-gravity work already implies.

…le (#364 follow-up)

Add WakeMotionRefinement (Swift + Kotlin twins), a post-pass over the
already-staged hypnogram that reclassifies a scored WAKE segment to light
when its per-minute step-tick cadence shows no locomotion and its per-minute
gravity posture stays stable outside isolated turn-over burst minutes (kept
as wake +/-1 min). Targets the HR-led wake call misreading a
hot-but-still/atonic stretch as an awakening -- #364 "Proposal 2": a real
anonymized night scored 194 min wake from single-minute turn-over bursts
(20-36 step-motion-counter ticks, a posture-variance spike) between minutes
of complete stillness, with zero walking cadence anywhere in the blocks.

Self-gates on OBSERVED gravity + step-sample density, never on strap
family/model (#345): a WHOOP 4.0 night fails the gate on the data itself
(gravity too sparse for a per-minute variance to mean anything, and no step
stream at all) and is left untouched every time; a WHOOP 5.0/MG night, which
streams both channels densely, is the expected beneficiary.

Default-off Experimental toggle (PuffinExperiment.motionAwareWake /
Settings -> Experimental -> Sleep staging on both platforms). The pass only
ever shrinks an already-scored wake segment -- it never invents wake time --
and detection plus the V1/V2 staging engines are untouched either way.

Tests (synthetic fixtures only): a hot-but-still night with a turn-over
burst every 90 min reclaims the still spans; a real 3-minute get-up stays
wake; a sparse (WHOOP-4.0-shaped) night leaves the gate declined; the
toggle's off-path is a byte-identical passthrough; five separate synthetic
patterns with different injected reclaim amounts are each recovered
exactly, proving the method tracks varying input rather than one fixture.
@ryanbr
ryanbr merged commit bd07471 into ryanbr:main Jul 14, 2026
10 checks passed
ryanbr pushed a commit that referenced this pull request Jul 14, 2026
…ores WAKE on a motionless wrist (fixes #462) (#465)

* fix(sleep): motion-corroborated wake — elevated HR alone no longer scores WAKE on a motionless wrist (#462)

Both stagers (SleepStager V1 and the default SleepStagerV2) and the HR-led
session confirmation (confirmSleepWithHR) called WAKE primarily off HR /
HR-variability with no motion or posture cross-check. On a night whose resting
HR is held elevated WITHOUT the wearer getting up — a supplement protocol, a
fever, a hot room, alcohol — that logic scored hot-but-motionless sleep as wake:
over-called WASO, mis-placed onset, tanked efficiency and Rest.

Rule: elevated HR ALONE is insufficient to call wake. An epoch or run at the
night's quiescent MOTION floor with UNCHANGED posture cannot be scored WAKE on
cardiac evidence alone. Corroboration comes from the gravity posture/jerk signal
both stagers already consume (always present), not step ticks.

- SleepStagerV2 (default): on a motion-quiescent epoch (no observed movement;
  peak jerk at/below the night-relative wake-gate floor) the AWAKE cardiac term
  is clamped to <= 0 — wake-SUPPRESSING (low, flat HR) evidence kept, the
  wake-PROMOTING half dropped. The clamp is applied over the restored (post-#437)
  0.8*zhvv + 0.4*zhrv coefficients, so a night with any motion stages
  byte-identical; only a still, elevated-HR epoch is held.
- confirmSleepWithHR (V1 detection): when a run is deeply motion-quiescent
  (>= ~90% of its dense-gravity minutes posture-stable, over >= 20 such minutes)
  the HR sleep band widens x1.05 -> x1.30. The band keeps a floor (genuine
  all-night in-bed wakefulness still dropped); with no gravity the strict band
  stands. runIsDeeplyQuiescent + posturVarianceG2 are pure and deterministic.
- adaptiveOvernightHRBaseline + sleepHRBaseline threading: an optional
  personalised overnight band (median of recent overnight medians, floored),
  threaded through detectSleep as an argument defaulting to nil (byte-identical
  when unset). Live cross-night wiring in IntelligenceEngine is a follow-up.

Why upstream #402 didn't cover this: #402 is a default-OFF post-pass over an
already-staged hypnogram, gated on step density. This acts where the
mis-scoring is produced (the stager emission + the HR confirmation), is on by
default, and reads gravity posture/jerk rather than step ticks.

Verification: `swift test` in Packages/StrandAnalytics is green — 1077 tests, 0
failures, including 8 new MotionCorroboratedWakeTests and every SleepStager /
SleepStagerV2 / WakeMotionRefinement golden suite (the byte-identical default
path with empty gravity / nil baseline is preserved). Two confirmed real nights
motivating the fix are documented in issue #462 (2026-07-13: 194 min WAKE vs
~67; 2026-07-14: onset 1:41 vs ~1:29 plus a 44-min WAKE block).

Kotlin twin deferred: the com.noop.analytics stagers need the same
motion-quiescent clamp and widened HR band transcribed to stay byte-identical
under the parity contract. Flagged as a Swift-only contribution (per PR #250);
tracked as a follow-up.

* feat(android): Kotlin twin of motion-corroborated wake (parity with PR #465)

Twin the StrandAnalytics motion-corroborated-wake fix (#462) into the Kotlin
analytics stagers:

- SleepStager.confirmSleepWithHR gains grav + sleepHRBaseline corroboration:
  a deeply motion-quiescent run (runIsDeeplyQuiescent / posturVarianceG2)
  widens the HR sleep band x1.05 -> x1.30 (quiescentHRSleepMult), floored so a
  genuinely awake high-HR still run is still dropped. adaptiveOvernightHRBaseline
  self-calibrates the band with a floor. sleepHRBaseline is threaded through
  detectSleep and folded into the memo key (byte-identical when unset / no gravity).
- SleepStagerV2.motionQuiescent clamps the AWAKE cardiac term to <=0 on a
  motion-quiescent epoch, keeping only wake-suppressing evidence; non-quiescent
  epochs stage byte-identical.
- MotionCorroboratedWakeTest mirrors the Swift fixtures (8 tests).

gradle: :app:testFullDebugUnitTest SleepStager* + MotionCorroboratedWakeTest =
85 tests, 0 failures (parity memo golden + V2 golden intact).
DX23876 pushed a commit to DX23876/noop that referenced this pull request Jul 22, 2026
…le (ryanbr#364 follow-up) (ryanbr#402)

Add WakeMotionRefinement (Swift + Kotlin twins), a post-pass over the
already-staged hypnogram that reclassifies a scored WAKE segment to light
when its per-minute step-tick cadence shows no locomotion and its per-minute
gravity posture stays stable outside isolated turn-over burst minutes (kept
as wake +/-1 min). Targets the HR-led wake call misreading a
hot-but-still/atonic stretch as an awakening -- ryanbr#364 "Proposal 2": a real
anonymized night scored 194 min wake from single-minute turn-over bursts
(20-36 step-motion-counter ticks, a posture-variance spike) between minutes
of complete stillness, with zero walking cadence anywhere in the blocks.

Self-gates on OBSERVED gravity + step-sample density, never on strap
family/model (ryanbr#345): a WHOOP 4.0 night fails the gate on the data itself
(gravity too sparse for a per-minute variance to mean anything, and no step
stream at all) and is left untouched every time; a WHOOP 5.0/MG night, which
streams both channels densely, is the expected beneficiary.

Default-off Experimental toggle (PuffinExperiment.motionAwareWake /
Settings -> Experimental -> Sleep staging on both platforms). The pass only
ever shrinks an already-scored wake segment -- it never invents wake time --
and detection plus the V1/V2 staging engines are untouched either way.

Tests (synthetic fixtures only): a hot-but-still night with a turn-over
burst every 90 min reclaims the still spans; a real 3-minute get-up stays
wake; a sparse (WHOOP-4.0-shaped) night leaves the gate declined; the
toggle's off-path is a byte-identical passthrough; five separate synthetic
patterns with different injected reclaim amounts are each recovered
exactly, proving the method tracks varying input rather than one fixture.
DX23876 pushed a commit to DX23876/noop that referenced this pull request Jul 22, 2026
…ores WAKE on a motionless wrist (fixes ryanbr#462) (ryanbr#465)

* fix(sleep): motion-corroborated wake — elevated HR alone no longer scores WAKE on a motionless wrist (ryanbr#462)

Both stagers (SleepStager V1 and the default SleepStagerV2) and the HR-led
session confirmation (confirmSleepWithHR) called WAKE primarily off HR /
HR-variability with no motion or posture cross-check. On a night whose resting
HR is held elevated WITHOUT the wearer getting up — a supplement protocol, a
fever, a hot room, alcohol — that logic scored hot-but-motionless sleep as wake:
over-called WASO, mis-placed onset, tanked efficiency and Rest.

Rule: elevated HR ALONE is insufficient to call wake. An epoch or run at the
night's quiescent MOTION floor with UNCHANGED posture cannot be scored WAKE on
cardiac evidence alone. Corroboration comes from the gravity posture/jerk signal
both stagers already consume (always present), not step ticks.

- SleepStagerV2 (default): on a motion-quiescent epoch (no observed movement;
  peak jerk at/below the night-relative wake-gate floor) the AWAKE cardiac term
  is clamped to <= 0 — wake-SUPPRESSING (low, flat HR) evidence kept, the
  wake-PROMOTING half dropped. The clamp is applied over the restored (post-ryanbr#437)
  0.8*zhvv + 0.4*zhrv coefficients, so a night with any motion stages
  byte-identical; only a still, elevated-HR epoch is held.
- confirmSleepWithHR (V1 detection): when a run is deeply motion-quiescent
  (>= ~90% of its dense-gravity minutes posture-stable, over >= 20 such minutes)
  the HR sleep band widens x1.05 -> x1.30. The band keeps a floor (genuine
  all-night in-bed wakefulness still dropped); with no gravity the strict band
  stands. runIsDeeplyQuiescent + posturVarianceG2 are pure and deterministic.
- adaptiveOvernightHRBaseline + sleepHRBaseline threading: an optional
  personalised overnight band (median of recent overnight medians, floored),
  threaded through detectSleep as an argument defaulting to nil (byte-identical
  when unset). Live cross-night wiring in IntelligenceEngine is a follow-up.

Why upstream ryanbr#402 didn't cover this: ryanbr#402 is a default-OFF post-pass over an
already-staged hypnogram, gated on step density. This acts where the
mis-scoring is produced (the stager emission + the HR confirmation), is on by
default, and reads gravity posture/jerk rather than step ticks.

Verification: `swift test` in Packages/StrandAnalytics is green — 1077 tests, 0
failures, including 8 new MotionCorroboratedWakeTests and every SleepStager /
SleepStagerV2 / WakeMotionRefinement golden suite (the byte-identical default
path with empty gravity / nil baseline is preserved). Two confirmed real nights
motivating the fix are documented in issue ryanbr#462 (2026-07-13: 194 min WAKE vs
~67; 2026-07-14: onset 1:41 vs ~1:29 plus a 44-min WAKE block).

Kotlin twin deferred: the com.noop.analytics stagers need the same
motion-quiescent clamp and widened HR band transcribed to stay byte-identical
under the parity contract. Flagged as a Swift-only contribution (per PR ryanbr#250);
tracked as a follow-up.

* feat(android): Kotlin twin of motion-corroborated wake (parity with PR ryanbr#465)

Twin the StrandAnalytics motion-corroborated-wake fix (ryanbr#462) into the Kotlin
analytics stagers:

- SleepStager.confirmSleepWithHR gains grav + sleepHRBaseline corroboration:
  a deeply motion-quiescent run (runIsDeeplyQuiescent / posturVarianceG2)
  widens the HR sleep band x1.05 -> x1.30 (quiescentHRSleepMult), floored so a
  genuinely awake high-HR still run is still dropped. adaptiveOvernightHRBaseline
  self-calibrates the band with a floor. sleepHRBaseline is threaded through
  detectSleep and folded into the memo key (byte-identical when unset / no gravity).
- SleepStagerV2.motionQuiescent clamps the AWAKE cardiac term to <=0 on a
  motion-quiescent epoch, keeping only wake-suppressing evidence; non-quiescent
  epochs stage byte-identical.
- MotionCorroboratedWakeTest mirrors the Swift fixtures (8 tests).

gradle: :app:testFullDebugUnitTest SleepStager* + MotionCorroboratedWakeTest =
85 tests, 0 failures (parity memo golden + V2 golden intact).
@vishk23
vishk23 deleted the motion-aware-wake branch July 30, 2026 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants