Sleep: V2 default for every strap, with a V1 revert toggle (reworks #347) - #351
Merged
Conversation
…revert toggle Reworks #347. #327 hard-gated WHOOP 4.0 to V1 (V2 over-stages on sparse motion, #319) and #347 would have removed the toggle entirely. But a 4.0 user (kavemang, #347) reported the opposite: V1 UNDER-stages badly — ~20-40 min deep+REM for an 8 h night (should be ~150-230) — and V2 fixed it. Neither engine is validated on 4.0, so forcing either is wrong; the honest answer is a sensible default plus an escape. - Drop the #327 family gate: the selector is now just the toggle on both the normal detected-night path and the self-heal restage (already toggle-driven), so V2 is the default for EVERY strap and a user can turn it off to fall back to V1. The toggle defaulted true already, so 5/MG is unchanged; 4.0 flips to V2. - Remove the now-trivial sleepStagerV2ForFamily helper + its family-gate test. - Fix the stale Android toggle comment; the Swift blurb already read correctly. Byte-parity: both platforms resolve useSleepStagerV2 straight from the toggle. Android compile + com.noop.analytics.* tests green; app-target Swift via app-build.
ryanbr
added a commit
that referenced
this pull request
Jul 14, 2026
…ver-calls awake in the field (#431) (#437) Reverts the #348 DREAMT re-tune's parameter values on both platforms. First healthy-sleeper field datapoint (#431): the same night, same raw runs, re-scored awake 6% -> 23% (30 min -> 1h47m), efficiency 94% -> 77%, dragging Rest ~-11 points and Charge with it — on every night in the log, not just one. The tune absorbed the DREAMT cohort's base rate into the population prior: DREAMT is a clinical OSA cohort (fragmented sleep, very high true WASO), and the shipped awake prior of 0.34 sits beside a doc comment still reading 'waso 10 %'. The no-regression guardrail was kappa (AAUWSS +0.03, Walch +0.006), and kappa doesn't guard stage-fraction calibration — a wake-fraction bias sails through. On a healthy sleeper the engine now starts every epoch from a 34%-wake prior with a 0.90 awake self-loop; the field signature follows. Kept from #348: the viterbi ln(max(v, 1e-9)) floor — inert for the restored matrix (no zero entries) but protects any future hand-edit from ln(0). The DREAMT cfg itself is not rejected — it can return behind a default-off Experimental toggle (the #351 V1/V2 precedent) once validated against healthy-sleeper ground truth (WHOOP-app awake-time comparisons). Verification: Android testFullDebugUnitTest = 2684 tests, 0 failures (SleepStagerV2Test golden + pins back on pre-#348 values); Swift twin is the symmetric revert, swift-packages CI covers StrandAnalytics.
DX23876
pushed a commit
to DX23876/noop
that referenced
this pull request
Jul 22, 2026
…revert toggle (ryanbr#351) Reworks ryanbr#347. ryanbr#327 hard-gated WHOOP 4.0 to V1 (V2 over-stages on sparse motion, ryanbr#319) and ryanbr#347 would have removed the toggle entirely. But a 4.0 user (kavemang, ryanbr#347) reported the opposite: V1 UNDER-stages badly — ~20-40 min deep+REM for an 8 h night (should be ~150-230) — and V2 fixed it. Neither engine is validated on 4.0, so forcing either is wrong; the honest answer is a sensible default plus an escape. - Drop the ryanbr#327 family gate: the selector is now just the toggle on both the normal detected-night path and the self-heal restage (already toggle-driven), so V2 is the default for EVERY strap and a user can turn it off to fall back to V1. The toggle defaulted true already, so 5/MG is unchanged; 4.0 flips to V2. - Remove the now-trivial sleepStagerV2ForFamily helper + its family-gate test. - Fix the stale Android toggle comment; the Swift blurb already read correctly. Byte-parity: both platforms resolve useSleepStagerV2 straight from the toggle. Android compile + com.noop.analytics.* tests green; app-target Swift via app-build.
DX23876
pushed a commit
to DX23876/noop
that referenced
this pull request
Jul 22, 2026
…tune over-calls awake in the field (ryanbr#431) (ryanbr#437) Reverts the ryanbr#348 DREAMT re-tune's parameter values on both platforms. First healthy-sleeper field datapoint (ryanbr#431): the same night, same raw runs, re-scored awake 6% -> 23% (30 min -> 1h47m), efficiency 94% -> 77%, dragging Rest ~-11 points and Charge with it — on every night in the log, not just one. The tune absorbed the DREAMT cohort's base rate into the population prior: DREAMT is a clinical OSA cohort (fragmented sleep, very high true WASO), and the shipped awake prior of 0.34 sits beside a doc comment still reading 'waso 10 %'. The no-regression guardrail was kappa (AAUWSS +0.03, Walch +0.006), and kappa doesn't guard stage-fraction calibration — a wake-fraction bias sails through. On a healthy sleeper the engine now starts every epoch from a 34%-wake prior with a 0.90 awake self-loop; the field signature follows. Kept from ryanbr#348: the viterbi ln(max(v, 1e-9)) floor — inert for the restored matrix (no zero entries) but protects any future hand-edit from ln(0). The DREAMT cfg itself is not rejected — it can return behind a default-off Experimental toggle (the ryanbr#351 V1/V2 precedent) once validated against healthy-sleeper ground truth (WHOOP-app awake-time comparisons). Verification: Android testFullDebugUnitTest = 2684 tests, 0 failures (SleepStagerV2Test golden + pins back on pre-ryanbr#348 values); Swift twin is the symmetric revert, swift-packages CI covers StrandAnalytics.
Merged
9 tasks
ryanbr
pushed a commit
that referenced
this pull request
Jul 28, 2026
… the claim is measurable (#925) * docs(sleep): say which stager actually ships — V2, not V1 Several comments on both platforms described `SleepStagerV2` as an "opt-in experimental" stager and V1 as "the default". That is backwards, and one Swift comment in `Repository.restageFromRaw` stated it outright: "V1 stays the default and is untouched". Both preferences are default ON: Strand/BLE/PuffinExperiment.swift experimentalSleepV2Enabled — `object(forKey:) == nil ? true : …` android/.../ble/PuffinExperiment.kt experimentalSleepV2 — `prefs.getBoolean(KEY, true)` Strand/Screens/SettingsView.swift @AppStorage(...) = true V2 was promoted over V1 in #277 and extended to every strap family in #351, so V2 is what stages a normal user's nights. The confusion has a single source: `useSleepStagerV2: Bool = false` in the `detectSleep` / `analyzeDay` signatures (SleepStager.swift:851, AnalyticsEngine.swift:328). That is the LIBRARY default, there so pure-function callers and the frozen-golden tests stay byte-identical. The app never takes it; the live call site threads the preference. Reading the signature alone gets the product's behaviour exactly wrong, so every one of these comments now names both defaults and says which is which. Verified empirically before changing anything, by replaying both stagers over one user's 36 recorded sessions and comparing epoch-for-epoch against the stored hypnogram. On the 21 UNEDITED nights (where the stored stages are whatever the live stager emitted), V2 reproduces the stored hypnogram byte-exactly on 21 of 21; V1 on 0 of 21, agreeing only 30.6–65.8%. The harness is `Tools/SleepBench`, added in the following commit, so anyone can re-run this against their own database. Comments only — no behaviour change on either platform. swift test (Packages/StrandAnalytics): 1167 tests, 0 failures ./gradlew assembleFullDebug testFullDebugUnitTest: 3201 tests, 0 failures, 5 skipped (JDK 17; counts read from app/build/test-results/testFullDebugUnitTest/TEST-*.xml, not from the wrapper's exit code) * tools(sleep): add SleepBench, an offline scoring harness for the stagers Eleven open PRs claim sleep-accuracy improvements and none of them is scored against a shared reference, so nobody can say which ones help or by how much. This adds the missing yardstick. `sleepbench --db <copy-of-a-noop-sqlite>` replays `SleepStager` (V1) and `SleepStagerV2` over every recorded night in a database and scores each hypnogram against whichever independent references that database carries: (a) hand-authored hypnograms — `sleepSession.userEdited` rows that ALSO carry a `stagelock:<device>:<startTs>` cursor; (b) the strap's own band sleep_state (the v18 @81 high nibble), which is an independent hardware verdict NOOP does not compute — and needs no human labelling, so it scales to a whole history for free; (c) the stored hypnogram itself, which identifies the recipe that produced it. Reported per stager: wake-minute error, sleep-onset and final-wake offsets, per-stage sensitivity/specificity, Cohen's kappa on both the 4-class and the sleep/wake problem, and per-night variance. PR #738's band veto is mirrored so its effect can be scored on the isolated staging path too. Two correctness points the harness makes rather than assumes: - `userEdited` alone does NOT mean human stage labels. The local editor corrects bed/wake BOUNDS and re-derives the hypnogram from raw, so its stages are machine output over a human-chosen window. Only a `stagelock` row had its stages authored directly. Section 0 reports the split so a caller cannot silently score a stager against its own output. - Session rows and stream rows can live under DIFFERENT deviceIds in a real database, so `--device` and `--stream-device` are separate arguments. Read-only by construction: the database is opened `SQLITE_OPEN_READONLY` with `immutable=1`, its path is always an argument, and nothing is ever written back. No health data lives in this repository. Built and run against a real 36-session database; `swift build -c release` clean.
ryanbr
added a commit
that referenced
this pull request
Jul 29, 2026
Two comment inaccuracies, no behaviour. The sleep files carried #600 and #690 from the project's pre-fork repo. Neither resolves here, and worse, both collide: this repo's #600 is an iOS Target Strain notification and its #690 is the body-location probe, which BodyLocationProbe cites correctly. So the same number meant two things depending on the file. Rewritten to references that resolve in THIS repo: - the V2 recipe keeps its credit as @sunny-noop, matching how the same contributor is credited in HealthExportPlan and SpotHrvReading, and drops the dead repo path - the V2-flag sites now cite 7.0.0, the release that introduced SleepStagerV2 here (CHANGELOG.md:240; the file arrived in commit 4da3cfb), plus #277/#351 for the default-ON claim, which those files already cite a line above - HrBroadcaster's credit keeps ryanbr, loses the dead path github.com/NoopApp/noop 404s and this repo is a fork whose parent no longer resolves, so every one of those links was dead. Nothing live pointed there: UpdateChecker and UpdateCheck both read api.github.com/repos/ryanbr/noop, and both About links go to github.com/ryanbr/noop. The two AppChangelog entries that mention NoopApp are left alone deliberately — they are the shipped release notes for the version that made that change, and rewriting them would falsify the record rather than correct it. Also: ReportReviewGate described the Oura Tier-B sidecars as capped at 20 MB. Both dumps enforce 25 MB (OuraActivityDump.maxBytes / OuraMotionDump.maxBytes).
ryanbr
added a commit
that referenced
this pull request
Jul 29, 2026
…#939) Two comment inaccuracies, no behaviour. The sleep files carried #600 and #690 from the project's pre-fork repo. Neither resolves here, and worse, both collide: this repo's #600 is an iOS Target Strain notification and its #690 is the body-location probe, which BodyLocationProbe cites correctly. So the same number meant two things depending on the file. Rewritten to references that resolve in THIS repo: - the V2 recipe keeps its credit as @sunny-noop, matching how the same contributor is credited in HealthExportPlan and SpotHrvReading, and drops the dead repo path - the V2-flag sites now cite 7.0.0, the release that introduced SleepStagerV2 here (CHANGELOG.md:240; the file arrived in commit 4da3cfb), plus #277/#351 for the default-ON claim, which those files already cite a line above - HrBroadcaster's credit keeps ryanbr, loses the dead path github.com/NoopApp/noop 404s and this repo is a fork whose parent no longer resolves, so every one of those links was dead. Nothing live pointed there: UpdateChecker and UpdateCheck both read api.github.com/repos/ryanbr/noop, and both About links go to github.com/ryanbr/noop. The two AppChangelog entries that mention NoopApp are left alone deliberately — they are the shipped release notes for the version that made that change, and rewriting them would falsify the record rather than correct it. Also: ReportReviewGate described the Oura Tier-B sidecars as capped at 20 MB. Both dumps enforce 25 MB (OuraActivityDump.maxBytes / OuraMotionDump.maxBytes).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reworks #347 per the discussion on that PR. Instead of removing the staging toggle and hard-gating WHOOP 4.0 to V1, this makes V2 the default for every strap and keeps a toggle to revert to V1.
Why
#327 hard-gated 4.0 → V1 (V2 over-stages on the 4.0's sparse motion, #319), and #347 would have removed the toggle entirely. But a 4.0 user (@kavemang, on #347) reported the opposite failure: V1 badly UNDER-stages — ~20–40 min of deep+REM for an 8-hour night (physiologically it should be ~150–230 min) — and switching to V2 fixed it.
So on 4.0, both engines are wrong for different users and neither is validated: V2 can over-stage (#319), V1 can catastrophically under-stage (#345/#347). Forcing either one is the wrong call. The honest answer is a sensible default plus an escape hatch.
What changed
sleepStagerV2ForFamilyhelper + its family-gate unit test (nothing left to unit-test — the selector is a bare toggle read).Trade-off (deliberate)
V2-default reintroduces the #319 over-staging risk for 4.0 users who don't touch the toggle. That's an accepted trade — V1's under-staging (kavemang) is worse for more users, and the toggle is the escape for anyone who prefers V1. The real fix is validating 4.0 staging against ground truth (#271/#319); until then this stops pretending either engine is "correct" for 4.0.
Parity / validation
useSleepStagerV2straight from the toggle (SwiftPuffinExperiment.experimentalSleepV2Enabled, KotlinuseExperimentalSleepV2).compileFullDebugKotlin+com.noop.analytics.*tests green.IntelligenceEngine) validated via app-build (running).Supersedes #347.