Skip to content

feat(vitals): per-version crash/ANR breakdown + stalled-feed warning - #25

Open
TimeToBuildBob wants to merge 3 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/vitals-version-breakdown
Open

feat(vitals): per-version crash/ANR breakdown + stalled-feed warning#25
TimeToBuildBob wants to merge 3 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/vitals-version-breakdown

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Two gaps surfaced by trying to answer "are crash rates better on v0.14.0b2?" (ActivityWatch/aw-android#176). Neither is answerable with the tooling as it stands.

1. The app-wide series can't attribute a rate to a release

data/android-crash-rate.csv is one userPerceivedCrashRate number per day across all installs. During a partial rollout that number is dominated by the install base still on the old build, so a genuinely fixed release barely moves the aggregate for weeks — and when it does move you can't tell the release from seasonality.

Adds vitals.py by-version, which requests the same metric with the API's versionCode dimension and compares versions directly:

uv run vitals.py by-version --days 14
uv run vitals.py by-version --kind anr-rate --as-csv

2. A stalled feed was invisible

_daily_freshness anchors each query to the API's latest available DAILY date. When that date stops advancing, the collector still exits 0 and rewrites identical rows — so git diff --cached --quiet skips the commit and the daily job keeps reporting success while the series silently stops.

That is not hypothetical: android-crash-rate.csv and android-anr-rate.csv were last updated 2026-08-22 (data through 08-19) while installed.csv updated today and every collect-play.yml run in that window is green.

_freshness_warning now prints the lag to stderr past STALE_FRESHNESS_DAYS (3), so a stall shows up as output rather than as absence of it. It does not fail the job — diagnosing whether this stall is Play-side or permissions-side needs the service account, which I don't have.

Notes

  • by-version prefers the API's valueLabel when it adds information, so rows read 1000 (0.14.0b2) rather than a bare code.
  • No behaviour change to the existing crash-rate / anr-rate / summary commands or the collected CSV format.
  • I have no Play service-account key, so the live API path is unverified; the query shape is covered by --dry-run and the parsing/warning logic by offline tests.

Verification

uv run ruff check ., uv run mypy *.py, and uv run pytest -q (11 passed) all green locally — 9 new tests in test_vitals.py covering the dimensioned query body, per-version row parsing, and the freshness warning.

Two gaps surfaced by 'are crash rates better on v0.14.0b2?'
(ActivityWatch/aw-android#176), which the existing tooling cannot answer.

1. The app-wide daily series cannot attribute a rate to a release. During a
   partial rollout the aggregate is dominated by the install base still on the
   old build, so a genuinely fixed version barely moves it for weeks. Add
   'vitals.py by-version', which breaks the same metric down by the API's
   versionCode dimension and compares versions directly.

2. A stalled feed was invisible. _daily_freshness anchors the query to the
   API's latest available DAILY date; when that date stops advancing the
   collector still exits 0, rewrites identical rows, and 'git diff --quiet'
   skips the commit — so the daily job reports success while the series
   silently stops. Warn on stderr when the feed is more than
   STALE_FRESHNESS_DAYS behind, so a stall is output rather than absence of it.

Add offline tests for the dimensioned query body, per-version row parsing, and
the freshness warning. ruff/mypy/pytest all green.
@greptile-apps

greptile-apps Bot commented Aug 26, 2026

Copy link
Copy Markdown

Greptile Summary

The PR adds per-version crash and ANR reporting and warns when Play’s daily vitals feed becomes stale.

  • Adds dimension-aware vitals queries and a by-version CLI command.
  • Groups results by raw version code while retaining labels for display.
  • Uses csv.writer for correctly escaped CSV output.
  • Adds offline coverage for query construction, parsing, grouping, CSV encoding, and freshness warnings.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the current implementation uses the raw version code as its grouping identity and delegates CSV field encoding to csv.writer.

Important Files Changed

Filename Overview
vitals.py Adds dimensioned reporting and stale-feed warnings; both previously reported issues are addressed at the current head.
test_vitals.py Adds focused offline tests for the new query, parsing, grouping, CSV, and freshness behavior.

Reviews (2): Last reviewed commit: "fix(tests): split imports and remove unu..." | Re-trigger Greptile

Comment thread vitals.py Outdated
Comment thread vitals.py Outdated
Two correctness issues found in Greptile review:

P1 (unstable version identity): `by-version` keyed `per_version` on the
display string from `_dimension_value`, which includes `valueLabel`. When Play
omits the label for some rows of the same versionCode the same release splits
into separate buckets, corrupting day counts and means. Fix: add
`_dimension_raw` that returns only the typed value (no label), group by that
as a stable key, and track the best display string in a separate dict.

P2 (unescaped CSV): the `--as-csv` branch interpolated version labels directly
into an f-string, so a comma or newline in a Play label would produce malformed
CSV. Fix: write through `csv.writer`.

Tests: 3 new tests covering `_dimension_raw`, stable-grouping behaviour, and
CSV escaping with a comma-bearing label (16 pass total).
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Fixed both Greptile findings in e50c0da (16 tests pass):

  • P1 stable version identity: added _dimension_raw as the grouping key (raw versionCode only, no label), version_labels dict tracks the best display string seen. Same versionCode with intermittent missing labels now lands in one bucket.
  • P2 CSV escaping: --as-csv now writes through csv.writer; commas/quotes/newlines in Play version labels are properly encoded.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI lint failure fixed in 617a113: split import csv, io as _io onto separate lines and removed the now-unused import io above it. 16 tests still passing.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI is green, both Greptile findings (P1 version-identity grouping, P2 CSV escaping) are fixed. Ready for maintainer review and merge.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

@greptileai review

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Current-head re-review is complete at 5/5 with no blocking findings; CI is green. I do not have merge permission on ActivityWatch/stats, so the only remaining action is maintainer review/merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant