devops: report CI test results as parquet artifacts#41669
Merged
Conversation
Adds a parquetReporter that writes each test result into an in-memory DuckDB and exports a parquet file per CI job, uploaded as an artifact for downstream compaction.
The .mts reporter is ESM; importing the named stripAnsi from the CommonJS ./utils failed at load with a missing-named-export error. Inline the tiny helper so the reporter is self-contained.
Docker runs tests inside the container, where the GITHUB_* vars weren't present, so every parquet row had null run/PR metadata. Forward them into the test-run exec.
npm 11 records libc fields for the duckdb native optional deps; CI runs npm 10 and strips them on install, failing the clean-tree check. Remove them so the committed lockfile matches CI.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
dgozman
reviewed
Jul 7, 2026
| - run: npm run build | ||
| - run: npx playwright install --with-deps | ||
| - run: npm run ct | ||
| - run: npm run ct -- --reporter=list,./tests/config/parquetReporter.mts |
Collaborator
There was a problem hiding this comment.
Why is this one different from others? Update the playwright config instead?
Member
Author
There was a problem hiding this comment.
This one doesn't have a config, but I also realised that we're not even emitting blobs for this, so I dropped it.
| "ignoreDeprecations": "6.0", | ||
| }, | ||
| "include": ["**/*.spec.js", "**/*.ts"], | ||
| "include": ["**/*.spec.js", "**/*.ts", "**/*.mts"], |
Collaborator
There was a problem hiding this comment.
Why make it .mts and not a .ts?
Member
Author
There was a problem hiding this comment.
So that the top-level await works. I did it because I didn't want to buffer all results to apply them in onEnd, but I thought again and realised that it's not as much memory as I thought. Turned it back into .ts.
Contributor
Test results for "tests 2"3 failed 36 flaky105909 passed, 4476 skipped Merge workflow run. |
Buffer test results and do all DuckDB work in onEnd, so the reporter can be a plain .ts file. Fix the docker artifact filename, drop redundant report_file inputs, tidy the upload action's required/default inputs, and drop parquet from the components lane where only coarse meta-runner rows were recorded.
dgozman
approved these changes
Jul 7, 2026
Contributor
Test results for "MCP"2 failed 7707 passed, 1235 skipped Merge workflow run. |
Contributor
Test results for "tests 1"3 flaky49501 passed, 1164 skipped Merge workflow run. |
This was referenced Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every CI test run now writes its results to a parquet file, uploaded as a per-job artifact. A separate compaction job (not in this PR) will ingest those into a queryable history - so we can answer "how flaky is this test, really" without scraping logs.
New reporter at
tests/config/parquetReporter.mtscollects results in an in-memory DuckDB and exports one parquet per run. Artifact retention is 7 days; the compaction job is the real system of record. CI only.