fix(import): store Oura efficiency as a 0-1 fraction, matching the native column

[vishk23]

What

Scope (updated — this PR grew beyond the original title): normalizes the efficiency column to a
single 0-1 fraction convention across both import lanes and both platforms:

1. The Oura API importer wrote Oura's native efficiency field — a 0-100 integer percent — straight
   into sleepSession.efficiency / dailyMetric.efficiency (OuraApiParser.swift).
2. The WHOOP CSV importer had the identical bug: "Sleep efficiency %" (0-100) written straight into
   the same 0-1 fraction columns, on both Swift and Kotlin.

NOOP's own sleep pipeline stores that shared column as a 0-1 fraction everywhere it computes it, so
rows from either importer ended up in the same column at two different scales, on both platforms.

Native convention (verified, not assumed)

Packages/StrandAnalytics/Sources/StrandAnalytics/AnalyticsEngine.swift:425
computes let efficiency = inBedS > 0 ? effWeighted / inBedS : 0.0 — a
ratio of two second-counts, always in [0,1] — and writes it directly
into both DailyMetric.efficiency (AnalyticsEngine.swift:698) and
CachedSleepSession.efficiency (AnalyticsEngine.swift:721).
Packages/StrandAnalytics/Sources/StrandAnalytics/SleepStageTotals.swift:112
computes the same ratio (total.asleep / total.inBed) independently. Real
WHOOP-era rows in an actual on-device database sit at 0.36-0.94; real
imported Oura rows sat at 83-96 — same column, two scales.

This inconsistency already left fingerprints in the UI, which is how it
was traced back to the importer rather than assumed:

• Strand/Screens/SleepView.swift:1173 — if e > 1.5 { e /= 100 } // efficiency arrives as % on some import paths
• Strand/Screens/SleepView.swift:2378-2380 — efficiencyPct(_:)
  defensively does stored <= 1.0 ? stored * 100 : stored
• Strand/Data/WatchSessionBridge.swift:198-202 — explicit comment:
  "efficiency is stored as a fraction in [0,1] in some paths and as a
  percent in others"

Both shims treat <= 1.0 as the correct/expected case, confirming the
fraction is the native convention, not the percent.

Fix

Oura API lane: normalize at the parse boundary in OuraApiParser.swift: divide by 100 where
efficiency is read off the session (WearableJSON.posDbl(s, "efficiency").map { $0 / 100.0 }). The day
rollup's efficiencyPct is derived from that same session value, so both the per-session row and the
daily rollup come out as fractions from one change.

WHOOP CSV lane (both platforms): the importer's "Sleep efficiency %" (0-100) is now divided at the
WRITE boundary — WhoopExportImporter.fractionFromImportedEfficiencyPct (Swift, used by
Strand/Data/WhoopImporter.swift) and WhoopCsvImporter.efficiencyFractionFromPct (Kotlin) — the same
shape as the existing Day Strain ⇄ Effort rescale, so the verbatim parsed percent value is preserved and
only the store boundary changes. The CSV exporters apply the inverse
(WhoopExportImporter.whoopEfficiencyPctFromFraction / the Kotlin twin in WhoopCsvExporter.kt) so an
exported CSV stays genuinely WHOOP-format and a NOOP→NOOP round-trip is lossless — rounded to 4 decimals
of a percent at emission, because num() prints shortest-round-trip Doubles and a raw fraction * 100
leaks FP dust (e.g. 92.30000000000001) into the cell otherwise.

OuraExportParser.swift (the separate Oura file-based account-export lane) reads the same raw
efficiency field independently and is not touched by this PR — it's a third lane with its own
percent/fraction question, left for a follow-up.

Data heal (migration, not just re-import) — now on BOTH platforms

Re-running an import wouldn't fix already-stored bad rows on its own: dailyMetric/sleepSession are
upserted with the importer's value on conflict, so simply re-syncing would overwrite a bad row with
another bad row until the parser fix above landed — a schema-level heal is required.

iOS/macOS (GRDB): Packages/WhoopStore/Sources/WhoopStore/Database.swift, migration
v26-efficiency-heal (appended after v25-oura-raw):

UPDATE sleepSession SET efficiency = efficiency / 100.0 WHERE efficiency > 1.5;
UPDATE dailyMetric  SET efficiency = efficiency / 100.0 WHERE efficiency > 1.5;

• UPDATE-only, no schema change.
• Unambiguous, idempotent threshold: no genuine fraction exceeds 1.0, and no genuine percent-scale
  value falls at or below 1.5, so the predicate can never touch an already-correct row, and a second run
  is a no-op.
• Not deviceId-scoped (broadened from the original deviceId = 'oura-api' version): both known
  percent writers — the Oura API importer's oura-api rows AND the WHOOP CSV importer's rows under
  whatever strap deviceId the user imported into — are healed by the same predicate.

Android (Room) — added in this PR, was previously missing: without this, an Android user's stored
efficiency would permanently diverge from an iOS user's for the same imported CSV (iOS heals via v26,
Android wouldn't), and worse, the new Android CSV exporter's efficiency * 100 would turn an unhealed
percent row (92) into 9200 in an exported CSV. android/app/src/main/java/com/noop/data/WhoopDatabase.kt,
migration MIGRATION_18_19 (bumps @Database(version = 18) → 19), the byte-parity twin of the GRDB
heal — same tables, same threshold, same non-scoped predicate:

UPDATE `sleepSession` SET `efficiency` = `efficiency` / 100.0 WHERE `efficiency` > 1.5
UPDATE `dailyMetric` SET `efficiency` = `efficiency` / 100.0 WHERE `efficiency` > 1.5

UPDATE-only, no schema/column change, so no schema-export/column-order concern (Room's exportSchema = false, no android/schemas/ in this repo). Pinned by a new plain-JVM test,
android/app/src/test/java/com/noop/data/EfficiencyHealMigrationTest.kt, following this codebase's
existing Room-migration-test convention (string-pinned SQL + version-pair assertions — there's no
Robolectric/JDBC-SQLite harness here to execute the migration against a real database).

Sequencing caveat (flagged in review, not this PR's fault): v26 (GRDB) collides with two other
pending PRs that also want the next GRDB slot, and Room's 18→19 slot likewise collides with another
pending Room migration. Whichever of the colliding PRs lands second will need to renumber — worth
sequencing deliberately rather than merging blind.

Android parity

The WHOOP CSV importer/exporter write-boundary fix and the Room heal above are now BOTH on Android in this
PR (see android/app/src/main/java/com/noop/ingest/WhoopCsvImporter.kt,
android/app/src/main/java/com/noop/ingest/WhoopCsvExporter.kt, and
android/app/src/main/java/com/noop/data/WhoopDatabase.kt).

The Oura API importer specifically has no Android equivalent to fix: android/'s com.noop.oura
package (OuraDriver.kt, OuraGatt.kt, Framing.kt, Decoders.kt, Auth.kt, RingGen.kt) is the BLE
ring protocol driver (headless, no android.bluetooth in the package itself) — a parallel, unrelated
feature for pairing directly with an Oura ring over Bluetooth, not a cloud API import path. No
ouraring.com/OAuth/cloud-import references exist in android/ outside that BLE driver, so there is no
Oura-API-shaped data on Android for the Room heal to worry about — only the WHOOP-CSV-shaped percent rows,
which the heal above covers.

Caveat — I have no Android toolchain in this environment (no JDK; ./gradlew can't even report
--version). The Kotlin write-boundary fix (efficiencyFractionFromPct / the exporter's round(it * 100.0 * 10_000) / 10_000) and the MIGRATION_18_19 heal + its test are careful transcriptions of the
Swift logic and this codebase's existing Room-migration style, reviewed closely but not compiled or
executed. testFullDebugUnitTest needs to run on a real toolchain before merge.

Tests

• Packages/StrandImport/Tests/StrandImportTests/OuraApiParserSleepTests.swift:
  testEfficiencyIsNormalizedToZeroToOneFraction — a raw "efficiency": 92
  input yields 0.92 on both the session and the day rollup.
• Packages/StrandImport/Tests/StrandImportTests/WhoopCsvExporterTests.swift:
  testEfficiencyPctFractionPairIsLosslessAndNilSafe — the fraction↔percent round-trip is lossless
  (including the FP-dust rounding), and the existing round-trip tests' fixtures now seed efficiency as
  the native 0-1 fraction.
• Packages/WhoopStore/Tests/WhoopStoreTests/MigrationTests.swift:
  testV26HealsEfficiencyPercentToFraction — seeds rows at the pre-v26 (v25-oura-raw) schema state via
  WhoopStore.makeMigrator().migrate(_:upTo:), then applies the rest of the migrator and confirms: an
  oura-api percent row heals, an already-fraction oura-api row is untouched (idempotent), a
  WHOOP-CSV-imported percent row under an arbitrary strap deviceId (my-whoop) ALSO heals (no longer
  deviceId-scoped), and a native fraction row under a strap deviceId is left alone.
• android/app/src/test/java/com/noop/ingest/WhoopCsvExporterTest.kt — its existing round-trip fixtures
  (cyclesRoundTripThroughRealParser etc.) now seed efficiency as the native 0-1 fraction too.
• New in this PR: android/app/src/test/java/com/noop/data/EfficiencyHealMigrationTest.kt — pins
  MIGRATION_18_19's version pair, exact SQL for both tables, that it's UPDATE-only (no schema
  mutation), and that the threshold/divisor/lack-of-deviceId-scoping matches the Swift heal.

Verification:

cd Packages/StrandImport && swift build && swift test
# Build complete
# Executed 200 tests, with 1 test skipped (unrelated, gated behind XIAOMI_REAL_DB) and 0 failures

cd Packages/WhoopStore && swift build && swift test
# Build complete
# Executed 257 tests, with 0 failures

No regressions in either package (re-verified on the current branch tip). Android: no toolchain available
here — testFullDebugUnitTest (including the new EfficiencyHealMigrationTest) needs to run on a real
build machine before merge; flagging honestly rather than claiming untested Kotlin as tested.

Honest verification

• The native-convention claim is grounded in the four file:line citations
  above (StrandAnalytics writer + two independent UI shims), not assumed
  from the bug description.
• TodayView.swift:4234's restCaption fallback formats
  d.efficiency with %.0f%% eff with no normalization at all (unlike
  the two shims cited above) — a narrower, rare-path inconsistency (it
  only fires when totalSleepMin is nil but efficiency isn't) left
  alone here as out of scope.
• The Android Room heal migration and its test are transcribed, not compiled/run locally — see the
  caveat under "Android parity" above.

[vishk23]

Pushed f1b0623 expanding this to the full efficiency-unit story after an audit of real data turned up the sibling: the WHOOP CSV importer wrote "Sleep efficiency %" (0–100) into the same 0–1 fraction columns — identical bug class, both platforms.

What the commit adds:

• Swift + Kotlin importers now divide at the write boundary (same shape as the Day Strain ⇄ Effort rescale, byte-parity helpers on both sides); parsed values stay verbatim.
• Both CSV exporters apply the inverse (×100) so exported CSVs are genuinely WHOOP-format and a NOOP→NOOP round-trip is lossless. The percent is rounded to 4 decimals at emission because num() prints shortest-round-trip Doubles — raw fraction*100 leaks FP dust (92.30000000000001) into cells.
• Migration broadened + renamed v26-efficiency-heal: the > 1.5 predicate now heals every percent-scale leftover regardless of deviceId, covering CSV imports under arbitrary strap ids as well as oura-api. (No genuine fraction exceeds 1.0; no genuine percent is ≤1.5 — still idempotent.)
• Tests: StrandImport 200/200, WhoopStore 257/257 (migration test now proves the strap-id percent row heals and a native fraction row is untouched), macOS app build clean.

Android caveats, honestly: the Kotlin changes are careful transcription (I have no Android toolchain here) — testFullDebugUnitTest needs a run on your side since android.yml is disabled. And healing Android's historical Room rows needs a maintainer-owned Room migration (schema-version bump + column-order pinning), deliberately not attempted blind; until then the boundary fix stops new corruption and the SleepImportedFiguresTest shim behavior still covers legacy-scale rows.

[ryanbr]

Excellent diagnosis and a comprehensive core fix — the efficiency scale is normalized at every boundary (Oura API import, WHOOP CSV import + export) on both platforms, byte-parity, and the native-convention evidence (AnalyticsEngine:425 / SleepStageTotals:112 + the existing <= 1.0-is-correct shims) is airtight. The GRDB heal (v26-efficiency-heal, WHERE efficiency > 1.5, no deviceId filter) correctly heals all percent writers, is idempotent, and is pinned by MigrationTests. I ran the Android CSV tests locally — green — and Swift package CI is green.

One thing I'd want closed before merge, though.

The Android Room heal isn't optional — and it compounds

You flag it in the migration comment ("No Android Room migration twin in this PR…"), but I don't think it can be deferred, for two reasons:

1. Stored-data parity violation. After this PR, an iOS user and an Android user who imported the same WHOOP CSV have different stored efficiency — iOS 0.92 (healed), Android 92 (unhealed). The parity contract is that stored data is byte-identical across platforms; a heal on GRDB but not Room breaks that.

2. It compounds through the new exporter — this is the sharp edge. The new Android exporter does efficiency * 100. On an unhealed percent row (92), that produces 9200 in the CSV's "%" column. So an Android user who exports their history after this lands gets garbage efficiency values — the write-fix + unhealed-row combination is actively wrong, not merely stale. (iOS is fine because its v26 heal runs first.)

The write-boundary fix is correctly twinned on both platforms; the heal needs to be too. Suggest adding the Room heal twin in this PR — a MIGRATION_n→n+1 running the same UPDATE … SET efficiency = efficiency / 100.0 WHERE efficiency > 1.5 on sleepSession + dailyMetric — or a committed follow-up issue landing immediately behind it, not a code-comment TODO.

Smaller notes

• Scope: the description says "this PR only touches the API-import path," but it also fixes WHOOP CSV import/export on both platforms — worth updating so a reviewer isn't surprised (it's a good broadening, just undersold).
• Migration coordination (heads-up, not this PR's fault): v26 collides with two other pending changes that also want the next GRDB slot, and the Android heal twin would want Room 18→19, which collides with another pending Room migration. Whichever lands second will need to renumber — worth sequencing deliberately.
• The > 1.5 threshold is a sound heuristic (no genuine fraction exceeds it, no realistic sleep-efficiency percent is below it); the theoretical ambiguity only sits in the unreachable 1.0–1.5 band.

Core fix is right and well-verified — just want the stored-data heal to land on both platforms so iOS and Android don't diverge (and so the Android export doesn't double-scale historical rows).

[vishk23]

Pushed 86ae44a adding the Room heal migration, per review — agreed it's required, not optional.

What changed:

• android/app/src/main/java/com/noop/data/WhoopDatabase.kt — @Database(version = 18) → 19; added MIGRATION_18_19 + EFFICIENCY_HEAL_MIGRATION_SQL, the byte-parity twin of WhoopStore's GRDB v26-efficiency-heal:

  UPDATE `sleepSession` SET `efficiency` = `efficiency` / 100.0 WHERE `efficiency` > 1.5
  UPDATE `dailyMetric` SET `efficiency` = `efficiency` / 100.0 WHERE `efficiency` > 1.5
  

  UPDATE-only, no schema/column change (confirmed sleepSession/dailyMetric @Entity(tableName=...) + efficiency: Double? columns directly against Entities.kt before writing this), so no schema-export or column-order concern applies here — exportSchema = false and there's no android/schemas/ directory in this repo, so there was nothing to regenerate.
• android/app/src/test/java/com/noop/data/EfficiencyHealMigrationTest.kt (new) — plain-JVM test following this repo's existing Room-migration-test convention exactly (e.g. DailySpo2RawMigrationTest, RrSeqMigrationTest): pins the 18→19 version pair, the exact SQL for both tables, that it's UPDATE-only with no ALTER/CREATE/DROP/INSERT/DELETE, and that the threshold (> 1.5)/divisor (/ 100.0)/lack of deviceId-scoping match the Swift heal. No Robolectric or JDBC-SQLite harness exists in this module (verified — no such test dependency in build.gradle.kts), so string-pinning is the established pattern here, not a shortcut I invented.
• gh pr edit 376 --body: rewrote the description's scope — it undersold what's actually in the PR now. Updated "What"/"Fix" to state both the Oura API import fix AND the WHOOP CSV import/export fix (both platforms), expanded "Data heal" to cover both the GRDB v26 heal and this new Room 18→19 heal, and rewrote "Android parity" (previously said "nothing to migrate on Android," which was true for the original Oura-only scope but stale after the CSV-lane commit).

Sequencing caveat (per your flag): this claims Room's next free slot (18→19) as of now. Same collision risk you called out for the GRDB v26 slot — if another pending PR also lands a Room migration first, whichever merges second needs to renumber. Carried this into the PR description so it's not lost.

What I could not verify: no Android toolchain in this environment — no JDK (./gradlew --version fails with "Unable to locate a Java Runtime"), so MIGRATION_18_19 and EfficiencyHealMigrationTest are careful transcription against the existing Migration/test style, not a compiled-and-run change. Please run cd android && ./gradlew testFullDebugUnitTest --tests "com.noop.data.EfficiencyHealMigrationTest" (and the full testFullDebugUnitTest for a regression check) on your side before merge.

Swift suites untouched by this commit; re-ran both anyway to confirm the branch tip is still green: StrandImport 200/200 (1 skipped by design), WhoopStore 257/257.

[tigercraft4]

Review (external, non-blocking). Fix logic (Oura efficiency as 0-1 fraction) looks right; scope growing to the WHOOP CSV lane is reasonable (same shared column). Two inline notes below.

[tigercraft4]

This Room migration is real and present — but per your own PR note it was "reviewed, not compiled or executed" (no Android toolchain available). Please confirm ./gradlew testFullDebugUnitTest (specifically EfficiencyHealMigrationTest) actually runs green before merge. Also flagging your own note about a possible Room migration-version-slot collision with other pending PRs — needs coordination with the maintainer on merge order.

[tigercraft4]

This comment says "No Android Room migration twin in this PR" — but this same PR adds exactly that twin (WhoopDatabase.kt EFFICIENCY_HEAL_MIGRATION_SQL, v18->v19). Looks like a leftover from an earlier draft; worth removing/updating so it doesn t mislead the next reader.