Skip to content

devops(test-results-db): CLI to compact CI test results into DuckDB#41673

Merged
Skn0tt merged 3 commits into
mainfrom
skn0tt-test-results-db-cli
Jul 8, 2026
Merged

devops(test-results-db): CLI to compact CI test results into DuckDB#41673
Skn0tt merged 3 commits into
mainfrom
skn0tt-test-results-db-cli

Conversation

@Skn0tt

@Skn0tt Skn0tt commented Jul 7, 2026

Copy link
Copy Markdown
Member

Builds on #41669 (per-job parquet report artifacts). Adds a small CLI in
utils/test-results-db/ plus a scheduled workflow that compacts those
per-run parquet reports into a single queryable DuckDB file.

  • download — fetch the latest maintained DuckDB artifact (starts from an empty db on the first run).
  • update — ingest parquet report artifacts from the last N days that aren't already stored, deduped by artifact_id, downloading + ingesting in parallel (tunable --concurrency).
  • truncate — drop the oldest runs so the db keeps at most N runs. At ~500 test-workflow runs/week, the default cap of 2000 covers roughly the last 4 weeks.
  • Scheduled workflow (every 3h) runs all three and uploads the test-results-db artifact. A playwright-test-results skill documents how to query it.

Trial-run end-to-end on CI: 215 artifacts → 345k rows in ~50s.

Example update run:

❯ GITHUB_TOKEN=$(gh auth token) node utils/test-results-db/cli.ts update
Test results database
 251685 rows from 153 artifacts

Scanning for new artifacts (last 7 days)
 100 new artifacts to import
 imported 16/100
 imported 32/100
 imported 48/100
 imported 64/100
 imported 80/100
 imported 96/100
 imported 100/100

Summary
 imported 100 new artifacts
 429958 rows from 29 runs
 size 33.3 MB

@Skn0tt Skn0tt requested a review from dgozman July 7, 2026 14:47
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Comment thread utils/test-results-db/cli.ts Outdated
Comment thread utils/test-results-db/db.ts Outdated
continue;
const id = String(artifact.id);
if (ingested.has(id)) {
if (++seen >= stopAfterSeen)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite understand this logic, but I guess we can give it a shot.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expect update to be run locally from machines with a not-so-great uplink to the Github API; and I want to make the minimum amount of API requests we need to ensure correctness. Fetching the entire last 7 days took 40s on my machine, which is too long. I'm using three characteristics of Github Artifacts to cut down work:

  1. the artifacts list API we're using here orders descending by artifact ID
  2. artifact IDs are auto-increment, and they're assigned upon upload time (not workflow start time)
  3. uploading an artifact is fast, so the time between the artifact ID being assigned, and it showing up in the artifacts list API, is very small

The fetching logic here combines this into a watermark like approach: Once we've seen 100 consecutive artifacts we already ingested, we assume all following pages are also fully ingested. On my machine, this trimmed down this phase from 40s to 3s.

@Skn0tt Skn0tt merged commit b8f840c into main Jul 8, 2026
44 of 46 checks passed
@Skn0tt Skn0tt deleted the skn0tt-test-results-db-cli branch July 8, 2026 08:17
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Test results for "MCP"

2 failed
❌ [chromium] › mcp/annotate.spec.ts:230 › should capture annotations via show --annotate @mcp-ubuntu-latest-chromium
❌ [firefox] › mcp/annotate.spec.ts:230 › should capture annotations via show --annotate @mcp-macos-latest-firefox

7707 passed, 1235 skipped


Merge workflow run.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

49510 passed, 1164 skipped


Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants