Skip to content

Server-side: /api/measurements/batch should upsert on stats:* externalIds #213

@MBombeck

Description

@MBombeck

Symptom (iOS-side, all stats:* cumulative metrics)

Today's Schritte tile freezes at whatever HK value the iPhone synced on the first batch-call of the day. Subsequent intra-day syncs see the same externalId already exists server-side, get status: "duplicate" back, and the value never updates. Same pattern would hit Active Energy, Sleep duration, Walking/Running Distance, Flights Climbed — any HK metric we send as a daily-aggregate row with a deterministic stats:HKQuantityTypeIdentifier...:YYYY-MM-DD external-id.

Investigated end-to-end on 2026-05-24 — full chain documented in iOS report at .planning/v056-marathon/v0628-C10-redux-report.md (iOS repo).

Root-cause chain

  1. iOS computes today's HK total (HKStatisticsCollectionQuery with .cumulativeSum, anchored at local-midnight).
  2. iOS POSTs the daily-stats row to /api/measurements/batch with externalId = "stats:HKQuantityTypeIdentifierStepCount:2026-05-24".
  3. Server checks (userId, type, externalId), finds the row already exists from this morning's sync, returns status: "duplicate". The new value is silently dropped — the existing row is not updated.
  4. Batch-response payload carries no row.id for the duplicate, so the iOS cache can't capture a serverMeasurementId and the mutation path stays on .repost (POST again next time) rather than .patch (which would target the right row by id).
  5. iOS-side cache writes lastPostedValue = newValue regardless of the duplicate verdict → on the next sync the cache sees lastPostedValue == newValue and skips the call entirely. The bug becomes invisible to the client.

Net: operator walks 5,000 steps after the first sync of the day → tile shows 300 (the first-sync value) until next midnight.

Why it's load-bearing

stats:* externalIds aren't a steps-only quirk — every cumulative HK metric uses the same daily-aggregate-row-per-day pattern. Fixing one pattern fixes Steps, Active Energy, Sleep duration, Walking/Running Distance, Flights Climbed, and every future cumulative metric on the same shape. Single highest-leverage server fix per the iOS code audit.

Existing iOS-side workaround

v0.6.1.25 shipped LiveHealthKitTodayStore (commits ba74b70b + b8784799) that reads HK live for today's Steps tile + chart-detail today-segment, bypassing the server-snapshot for that one cell. Historical days still depend on the server snapshot, so cross-device parity (web ↔ iOS) for days other than today is still broken until the server fix lands.

Two clean fix options

A) Upsert on stats:* externalIds (recommended)

Make /api/measurements/batch treat duplicates on externalId starting with stats: as an update (overwrite value, bump updatedAt), not as a discard. Limited to the stats: prefix to avoid changing semantics for sample-class measurements where every reading is canonical and de-dup is desirable.

  • Pro: client semantics stay identical; iOS keeps re-posting and the right value lands; no client refactor needed.
  • Pro: fixes all stats:* metrics simultaneously.
  • Con: needs an audit-log entry for the overwrite (measurement.value.updated instead of measurement.duplicate-skipped) so historical revisions are traceable.

B) Batch response carries row.id for duplicates

When the server returns status: "duplicate", include the existing row's id in the response payload. iOS captures it into serverMeasurementId on first sync and switches the per-day mutation path to PATCH /api/measurements/<id> for subsequent updates.

  • Pro: no semantic change to batch behavior. Pure response-shape addition.
  • Pro: gives iOS a uniform handle for any future per-day overwrite use-case.
  • Con: requires an iOS-side refactor to route mutations through PATCH for stats:* ids — that's still an iOS ship to follow the server change.

Selected: A. Cleaner contract, single-PR fix, no iOS follow-up needed.

Acceptance criteria

  • POST /api/measurements/batch with an existing stats:HKQuantityTypeIdentifierStepCount:YYYY-MM-DD row + a higher value returns status: "updated" (or equivalent non-duplicate verdict) and the row's value reflects the new number on a subsequent GET.
  • New audit-log event measurement.batch.stats-overwrite surfaces in the wide-event stream so duplicate-vs-overwrite is auditable.
  • iOS-side: walking 1,000 steps after a same-day sync produces a fresh value in the Schritte tile within 30 s of the next .active scenePhase.

Reference

  • Existing wide-event annotation pattern: dashboard.widgets.validation-failed is the template

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions