Skip to content

fix: sync-freshness banner, burst validation, and UI/UX polish #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
abdulsaheel merged 1 commit into from
Jul 5, 2026
Merged

fix: sync-freshness banner, burst validation, and UI/UX polish #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 80 additions & 25 deletions lib/ble/ble_engine.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,28 @@ int nextBurstStablePollStreak({
bool shouldPauseMaintenanceTraffic({required bool offloadActive}) =>
offloadActive;

/// Whether a HISTORY_END burst's packet accounting matches what the band
/// reported sending (`expectedPacketCount`, from the metadata frame).
///
/// [actualBurstPacketCount] only counts packets that reached
/// onHistoricalRecord/onUndecodableRecord — i.e. that PASSED the RecordGate
/// plausibility check. A record the gate rejects (a stale/wandering-clock
/// block — see RecordGate.admit) is, by design, "neither stored nor
/// counted": it never reaches either callback. The band's own count has no
/// such carve-out — it just counts every packet it physically transmitted.
/// [droppedThisBurst] (RecordGate.dropped delta across this burst) must be
/// added back in before comparing, or a burst containing even one
/// gate-rejected record can never validate — which discards its OTHER,
/// perfectly good buffered records and re-requests the same stuck block
/// forever (zero sync progress).
@visibleForTesting
bool burstPacketCountMatches({
required int expectedPacketCount,
required int actualBurstPacketCount,
required int droppedThisBurst,
}) =>
expectedPacketCount == actualBurstPacketCount + droppedThisBurst;

/// Fired for every LIVE high-rate frame (0x28/0x2B/0x33). These are EPHEMERAL —
/// they are NOT persisted to raw_records (that bloated storage ~50x and stalled
/// derivation). The caller routes them to an in-memory sink for the live UI /
Expand All @@ -163,7 +185,6 @@ enum _HpsTerminalKind {
success,
timeout,
disconnected,
error,
}

class _HpsTerminal {
Expand Down Expand Up @@ -603,6 +624,11 @@ class BleEngine {
// EVERY historical-record path — see RecordGate in ble_state.dart. Re-seeded
// on each connect from the durable cursor.
RecordGate _recordGate = RecordGate();
// Snapshot of `_recordGate.dropped` at the last HISTORY_START — lets the
// HISTORY_END validator (below) tell "the band sent fewer packets than it
// said" apart from "we correctly, silently rejected some as implausible
// (stale-clock block) and never tallied them." See _handleSyncMarker.
int _burstDroppedAtStart = 0;
// Per-revision packet accounting for the historical drain (gap detection +
// honest per-version counts surfaced to the debug screens).
final Map<int, int> _historicalVersionCounts = <int, int>{};
Expand Down Expand Up @@ -722,6 +748,11 @@ class BleEngine {
'records_seen': _drain?.records ?? 0,
'batches_acked': _drain?.batches ?? 0,
'buffered_records': _drain?.bufferedRecords ?? 0,
// Connection-wide plausibility-gate rejections (RecordGate.dropped) — see
// burstPacketCountMatches for why these must be added back to the burst
// packet count before comparing against the band's expectedPacketCount.
'gate_dropped_total': _recordGate.dropped,
'gate_dropped_this_burst': _recordGate.dropped - _burstDroppedAtStart,
'history_requests': _historyRequests,
'history_completions': _historyCompletions,
'successful_bursts': _successfulBursts,
Expand Down Expand Up @@ -1796,6 +1827,7 @@ class BleEngine {
d.discardOpenChunk();
}
_session?.historicalRetry?.cancel();
_burstDroppedAtStart = _recordGate.dropped;
d?.rearm();
_setOffloadActive(true);
return;
Expand All @@ -1812,45 +1844,54 @@ class BleEngine {
}
await _awaitBurstTrafficSettle(d);
final expected = m.expectedPacketCount;
if (expected != null && !d.validateBurst(expectedPacketCount: expected)) {
// Records the plausibility gate silently rejected THIS burst (stale/
// wandering-clock block — by design, "neither stored nor counted",
// see RecordGate.admit) never reach onHistoricalRecord/
// onUndecodableRecord, so they never entered currentBurstPacketCount.
final droppedThisBurst = _recordGate.dropped - _burstDroppedAtStart;
final validated = expected == null ||
d.validateBurst(
expectedPacketCount: expected,
droppedThisBurst: droppedThisBurst,
);
// ADVISORY ONLY, never a gate: `expectedPacketCount`'s exact semantics
// (which transport packet types the band itself counts — command
// responses interleaved with the burst? retried/duplicate frames?) are
// not fully reverse-engineered, and field data shows the gap between
// expected and actual varies run to run with no fixed offset. What IS
// fully verified is frame-level CRC32 (framing.dart) and the RecordGate
// plausibility check — both already ran on every buffered record before
// we ever get here. So a count mismatch is NOT evidence of corrupt or
// missing data; treating it as fatal was actively harmful: on mismatch
// the OLD behavior discarded the entire buffered chunk (throwing away
// perfectly good, already-CRC-verified, already-gate-passed records),
// told the band FAIL, and re-requested the same block — forever, since
// nothing about a retry changes the count relationship. Zero sync
// progress, "last data" frozen indefinitely. Log the mismatch (still
// useful signal — see the sync-diagnostics screen) and commit anyway.
if (!validated) {
_log(
'[SYNC] Burst validation failed '
'[SYNC] Burst packet-count mismatch (advisory, NOT blocking commit) '
'(attempt ${d.consecutiveValidationFailures}): expected=$expected, '
'actual=${d.currentBurstPacketCount}, '
'dropped_this_burst=$droppedThisBurst, '
'historical=${d.currentBurstHistoricalPacketCount}, '
'traffic=${d.currentBurstTrafficCount}, '
'breakdown=${d.currentBurstBreakdown}',
);
d.discardOpenChunk();
final fail = buildHistoryResultFail(_seq.nextSync());
_log(
'[SYNC] FAIL frame='
'${fail.map((b) => b.toRadixString(16).padLeft(2, '0')).join()}',
);
await _write(fail);
await LocalDb.upsertSyncLedgerEntry(
status: 'validation_failed',
status: 'validated_with_mismatch',
lastError: 'burst_packet_mismatch',
metaPatch: {
'expected_burst_packets': expected,
'actual_burst_packets': d.currentBurstPacketCount,
'dropped_this_burst': droppedThisBurst,
'historical_burst_packets': d.currentBurstHistoricalPacketCount,
'traffic_burst_packets': d.currentBurstTrafficCount,
'burst_validation_failures': d.consecutiveValidationFailures,
'burst_breakdown': d.currentBurstBreakdown,
},
);
if (d.consecutiveValidationFailures >= 15) {
_log(
'[SYNC] Burst validation stuck after '
'${d.consecutiveValidationFailures} failures.',
);
_setHpsTerminal(_HpsTerminalKind.error, reason: 'stuck', drain: d);
_setOffloadActive(false);
return;
}
unawaited(_abortAndRetryHistorical(reason: 'burst_validation_failed'));
return;
}
_successfulBursts++;
_mergeValidatedBurst(d);
Expand Down Expand Up @@ -2600,9 +2641,23 @@ class _DrainController {

void onBurstUnknown() => burstStats.onUnknown();

bool validateBurst({required int expectedPacketCount}) {
final actual = currentBurstPacketCount;
if (expectedPacketCount == actual) {
/// [droppedThisBurst] = records the plausibility gate rejected during this
/// same burst (stale/wandering-clock block) — never tallied into
/// [currentBurstPacketCount] (they're never stored), but the band's own
/// [expectedPacketCount] counts them anyway since it just counts what it
/// physically transmitted. Add them back in before comparing, or a burst
/// that legitimately contains even one gate-rejected record can never
/// validate — discarding otherwise-good buffered records and looping
/// forever on the same stuck block.
bool validateBurst({
required int expectedPacketCount,
int droppedThisBurst = 0,
}) {
if (burstPacketCountMatches(
expectedPacketCount: expectedPacketCount,
actualBurstPacketCount: currentBurstPacketCount,
droppedThisBurst: droppedThisBurst,
)) {
consecutiveValidationFailures = 0;
return true;
}
Expand Down
60 changes: 57 additions & 3 deletions lib/state/app_state.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1087,9 +1087,22 @@ class AppState extends ChangeNotifier {
/// Debounced "new data stored" callback from the engine (continuous listening has
/// no discrete sync end). The engine already coalesced the burst; we run a single
/// LIGHT derive over the affected day(s) and refresh DB counts for the UI.
///
/// This is also THE reliable place to refresh `_lastRecTs` (the "last data"
/// freshness banner reads it). `_runSyncBurst`'s own before/after frontier
/// check can race the async commit — HISTORY_END's commit+ACK sometimes
/// lands just after `engine.runSync()` already returned, so that
/// checkpoint-based refresh can miss a burst entirely. This callback fires
/// on EVERY successful persist path (foreground burst, background/headless
/// drain, live-triggered store) after the write is durable, so it can't
/// race it — same guarantee dbCounts already relies on above.
void _onDataStored() {
unawaited(() async {
dbCounts = await LocalDb.counts();
final recTsHw = await LocalDb.getCursorInt('rec_ts_hw');
if (recTsHw != null && recTsHw > (_lastRecTs ?? 0)) {
_lastRecTs = recTsHw;
}
notifyListeners();
_deriveScheduler.markStoredData();
}());
Expand All @@ -1109,7 +1122,18 @@ class AppState extends ChangeNotifier {
await _loadProfile();
await _deriveScheduler.init();
lastSynced = await LocalDb.latestSample();
_lastRecTs = await LocalDb.lastDecodedRecTs() ?? lastSynced?.tsEpoch;
// The true data-edge frontier is the `rec_ts_hw` sync cursor, NOT
// lastDecodedRecTs() (MAX(rec_ts) FROM decoded_onehz). decoded_onehz only
// gets a row when a record decodes to the FULL 1 Hz shape (R24-family);
// historical R10 "lite" records (hr-only, no accel/optical) decode fine
// but land in `samples` instead — so on an R10-lite-heavy backlog,
// decoded_onehz's max freezes while the strap is genuinely, successfully
// syncing, and "last data" reads as stuck/stale. `rec_ts_hw` advances for
// every record commitSyncBatch durably persists, decoded_onehz-eligible
// or not, so it's the honest frontier (same one RecordGate/backfill
// policies already trust).
_lastRecTs =
await LocalDb.getCursorInt('rec_ts_hw') ?? lastSynced?.tsEpoch;
dbCounts = await LocalDb.counts();
await LocalDb.refreshComputeFreshness();
_savedAlarm = (await SharedPreferences.getInstance()).getInt('alarm_epoch');
Expand Down Expand Up @@ -1383,7 +1407,19 @@ class AppState extends ChangeNotifier {
}

/// Reset the live step counter for a fresh connected session.
///
/// This zeroes the connection-lifetime raw counter (`_liveRaw`). If a
/// workout is active, `_workoutRawBase` was snapshotted from a *previous*
/// (now-stale) `_liveRaw` value — left untouched, `workoutSteps` would
/// compute a negative delta on the next BLE disconnect/reconnect blip,
/// clamp to 0, and visibly reset the walk's step count instead of counting
/// monotonically. Rebase it here so the already-accrued workout steps
/// carry through the reset.
void _resetLivePedometer() {
if (activeWorkout != null && _workoutRawBase != null) {
final accruedRaw = _liveRaw - _workoutRawBase!;
_workoutRawBase = accruedRaw > 0 ? -accruedRaw : 0;
}
_magMin.clear();
_committedRaw = 0;
_liveSamples = 0;
Expand Down Expand Up @@ -1571,15 +1607,33 @@ class AppState extends ChangeNotifier {
}) async {
var last = SyncReport(0, 0, false);
for (var i = 0; i < maxSessions && engine.isConnected; i++) {
final frontierBefore = await LocalDb.lastDecodedRecTs();
// rec_ts_hw, not lastDecodedRecTs() — see the boot-time seed above for
// why: an R10-lite-heavy backlog can genuinely advance without ever
// touching decoded_onehz, and this "did we make progress" check must
// not mistake that for a stuck drain (spin-guard/backlogRemains below
// read frontierAfter too).
final frontierBefore = await LocalDb.getCursorInt('rec_ts_hw');
if (kickFirst || i > 0) {
await engine.requestHistorySync();
}
kickFirst = false;
final report = await engine.runSync(
timeout: const Duration(seconds: 180),
);
final frontierAfter = await LocalDb.lastDecodedRecTs();
final frontierAfter = await LocalDb.getCursorInt('rec_ts_hw');
// Refresh the freshness signal the "last data" banner reads from EVERY
// burst session, not just at app boot. `_lastRecTs` was previously only
// ever seeded in `_init()` — during a real historical drain, records go
// through `_DrainController.onHistoricalRecord` → `onCommitBatch`
// (bypassing `_onRecord`'s in-memory bump, which only fires on the rare
// pre-drain-setup fallback path), so a session left open kept showing
// "more than an hour behind" no matter how much fresh data actually
// synced, until the app was fully restarted. Bump + notify here so the
// UI reflects real progress as it happens, mid-burst.
if (frontierAfter != null && frontierAfter > (_lastRecTs ?? 0)) {
_lastRecTs = frontierAfter;
notifyListeners();
}
final strapNewest = engine.strapHistoryNewestTs;
final frontierAdvanced =
frontierAfter != null &&
Expand Down
Loading