Skip to content

Sleep stages: recognise both spellings of wake in segment comparisons (#979) - #1011

Merged
ryanbr merged 2 commits into
mainfrom
fix/stage-vocabulary-wake-awake
Aug 1, 2026
Merged

Sleep stages: recognise both spellings of wake in segment comparisons (#979)#1011
ryanbr merged 2 commits into
mainfrom
fix/stage-vocabulary-wake-awake

Conversation

@ryanbr

@ryanbr ryanbr commented Aug 1, 2026

Copy link
Copy Markdown
Owner

From the "also recorded" note in #988, filed separately as it asked.

Stored hypnograms carry both wake and awake. Five segment comparisons on each platform recognised only one of them — and they are the same five sites, because the Kotlin reimplementation reproduced the bug faithfully:

Swift Kotlin
SleepStager.swift:1075, :2193 SleepStager.kt:1157, :2378
WakeMotionRefinement.swift:188 WakeMotionRefinement.kt:223
WearableImporter.swift:134 WearableExportImporter.kt:788
XiaomiImporter.swift:136 XiaomiBandImporter.kt:335

What goes wrong

The damaging shape is stage != "wake" used to mean asleep. An imported "awake" segment falls straight through it and is counted as sleep, inflating the efficiency figure. The mirror shape, stage == "wake", under-counts wake time — and made #987's wake refinement skip those segments entirely, so the transition rule that shipped in 9.3.0 was silently not applying to imported nights that use the other spelling.

Six other sites already defend with case "wake", "awake", and Repository.swift:1339 / WhoopRepository.kt:733 get it exactly right. That is what makes this a missing shared rule rather than a missing idea.

Why a predicate and not a canonicaliser

There are two vocabularies, by design, and neither moves here:

  • Segment stage strings canonicalise to "wake"SleepStagerV2 models its own states as "awake" internally and renames on the way out, saying so in its stage-mapping comment.
  • Minutes-dictionary keys canonicalise to "awake" (SleepStageTotals, SleepWindowReclip).

The bug is the dictionary vocabulary reaching a segment comparison, which happens because imports never pass through SleepStagerV2 — Oura's phase table is ["deep","light","rem","awake"], and generic wearable JSON carries whatever the source app wrote.

So SleepStageVocabulary.isWake(_:) is a predicate. It fixes the comparisons without rewriting a single stored string, so no persisted hypnogram changes meaning. SleepStagerV2's own == "awake" tests are deliberately untouched — that is its internal model vocabulary, and changing it would move stored output.

Parity

Twin helpers, twin tests, same cases in the same order. Android's UI canonicalStage() now folds through the shared predicate so the alias rule has one definition; it still returns "awake", because that is the key its colour table and the minutes dictionaries use.

Worth recording: the two platforms canonicalise in opposite directions — Android's UI folds wake -> awake, Swift's SleepStagerV2 folds awake -> wake. Harmless today because each is internal, but they are each other's alias, which matters if a canonical stage string ever crosses the .noopbak boundary. Not changed here; that decision deserves its own PR.

Verification

  • 6 new tests per platform. The Kotlin suite runs green locally against the object as written: OK (6 tests). The Swift twin runs in swift-packages CI. Cases: both spellings, sleep stages excluded, casing/whitespace folded, unknown/empty not wake, plus the two regressions in the shapes the callers actually use — an awake + deep night counting only deep as asleep, and a wake total including both spellings.
  • Kotlin: full src/main/java compile, 2517 errors on this branch and 2517 on main, error sets byte-identical. No new errors.
  • swiftc -parse clean on every changed Swift file and the new test.
  • Tools/doc_comment_lint.py clean.
  • Swept both platforms afterwards: no single-spelling segment comparison remains, and the only surviving matches are inside the new helper's own implementation and doc comment.

Correction: this is latent, not active — no release note needed

My first draft of this section warned that sleep numbers would move for imported nights, "Oura in particular". I went looking for the affected nights and could not find any, so that warning was wrong and is withdrawn.

No producer on either platform writes "awake" into a stored segment. NOOP's own scorers emit "wake" (SleepStagerV2 renames its internal awake on output; V1 writes wake). XiaomiImporter maps .awake/.awakeInBed/.unknown to "wake". FitbitExportParser folds "wake", "awake" and "restless" to "wake". Oura's "awake" appears in a doc comment describing the upstream wire format, not in anything stored. The only "awake" segment in the tree is a test fixture — which is itself evidence that someone expected it reachable.

So no stored night changes today, and there is nothing to put under "scores that change".

That does not make the five sites correct. They are wrong, and the spelling arrives the moment anything bypasses the normalising importers — a new importer that passes its source vocabulary through, a hand-edited hypnogram, or #746's row-copy import of a backup from another fork, which copies segments verbatim from a tree whose vocabulary we do not control. This closes the hole before that lands rather than after.

The honest summary: correctness hardening with a real failure mode and, as of today, no behavioural blast radius.

app-build.yml is disabled, so nothing compiles the two Strand/Data importers — those edits are parse-checked only. The analytics half and both test suites are covered by CI.

Re-review: the tests did not test the fix

The six predicate tests pass whether or not the five call sites were actually changed — they exercise the rule, not its users. For a latent bug that matters more than usual: nothing else would ever notice a reverted site, because no production data reaches it today.

Each platform now also drives a real caller. SleepStager.hypnogramMetrics over the same night as the existing AASM test, with the WASO segment spelled awake:

  • tst is computed from a positive list (light || deep || rem), so it is immune either way and pins at 1080 s — that is also why my sweep did not need to touch it.
  • wasoS and disturbances are not immune. They read 60 / 1 with the fix and 0 / 0 without it, so a reverted call site fails the suite.

I checked the two implementations agree before writing the expectations rather than assuming parity: onset, sptEnd and the clip arithmetic are identical in Swift and Kotlin.

Also confirmed this pass: SleepStager.swift:2193 feeds WASO and the disturbance count, not an internal total — so the mirror shape was under-reporting a user-visible number, not just a private one. And the Kotlin site at WearableExportImporter.kt:788 is genuinely efficiencyFromStages, the same computation as Swift's efficiency(), so the parity claim in the table above is verified rather than assumed from the line numbers.

ryanbr added 2 commits July 31, 2026 19:57
…#979)

Stored hypnograms carry both 'wake' and 'awake'. Five segment comparisons on
EACH platform recognised only one, and they are the same five sites — the
reimplementation reproduced the bug faithfully:

  SleepStager 1075/2193      <-> SleepStager.kt 1157/2378
  WakeMotionRefinement 188   <-> WakeMotionRefinement.kt 223
  WearableImporter 134       <-> WearableExportImporter.kt 788
  XiaomiImporter 136         <-> XiaomiBandImporter.kt 335

The damaging shape is `stage != "wake"` used to mean asleep: an imported
'awake' segment falls through it and is counted as SLEEP, inflating the sleep
efficiency figure. The mirror shape `stage == "wake"` under-counts wake time,
and made #987's wake refinement skip those segments entirely.

Six other sites already defended with `case "wake", "awake"`, which is what
makes this a missing shared rule rather than a missing idea.

Two vocabularies exist by design and neither moves here: segment strings
canonicalise to 'wake', minutes-dictionary keys to 'awake'. The bug is the
dictionary vocabulary reaching a segment comparison, which happens because
imports do not pass through SleepStagerV2 — Oura's phase table is
[deep, light, rem, awake]. So this is a PREDICATE, not a canonicaliser: it
fixes the comparisons without rewriting any stored string, and no persisted
hypnogram changes meaning.

SleepStagerV2's own "awake" tests are deliberately untouched; that is its
internal model vocabulary, renamed to 'wake' on output.

Android's UI canonicalStage() now folds through the shared predicate so the
alias rule has one definition. It still returns 'awake', because that is the
key its colour table and the minutes dictionaries use.

6 twin tests per platform, same cases in the same order.
The six predicate tests pass whether or not the five call sites were
actually changed - they exercise the rule, not its users. Nothing in the
suite noticed a reverted site, which for a latent bug is the only way a
regression would ever surface.

Each platform now also drives a real caller: SleepStager.hypnogramMetrics
over the same night as the existing AASM test, with the WASO segment spelled
'awake'. tst comes from a positive list (light/deep/rem) so it is immune
either way at 1080 s; WASO and the disturbance count are not, and read 0/0
before the fix.

Verified the two implementations agree before asserting: onset, sptEnd and
the clip arithmetic are identical in Swift and Kotlin, so both expect
waso 60 and disturbances 1.
@ryanbr
ryanbr merged commit 310cecc into main Aug 1, 2026
14 checks passed
@ryanbr
ryanbr deleted the fix/stage-vocabulary-wake-awake branch August 1, 2026 03:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant