Conversation
- Introduced Codecov configuration in `codecov.yaml` for managing coverage uploads. - Updated `.gitignore` to exclude Vitest coverage reports. - Added `test:coverage` scripts in package.json files for all packages. - Created Vitest configuration files for coverage reporting in `packages/arkor`, `packages/cli-internal`, `packages/create-arkor`, and `packages/studio-app`. - Enhanced CI workflow to include a coverage job that uploads reports to Codecov.
- Added @vitest/coverage-v8 version 4.1.5 to multiple package dependencies in pnpm-lock.yaml. - Updated vitest version entries to reflect the new coverage dependency across relevant packages.
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. ℹ️ You can also turn on project coverage checks and project coverage reporting on Pull Request comment Thanks for integrating Codecov - We've got you covered ☂️ |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 44bd6521b4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }, | ||
| "devDependencies": { | ||
| "@types/node": "^24", | ||
| "@vitest/coverage-v8": "^4.1.5", |
There was a problem hiding this comment.
Adding @vitest/coverage-v8 to workspace package.json files without updating pnpm-lock.yaml makes CI installs fail under pnpm’s default frozen-lockfile behavior in CI. Running pnpm install --frozen-lockfile --lockfile-only now errors with ERR_PNPM_OUTDATED_LOCKFILE (first reported for packages/cli-internal/package.json), so both the existing build job and the new coverage job will fail before tests run.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Pull request overview
Adds per-package Vitest v8 coverage generation across the monorepo and introduces a dedicated CI job to upload those LCOV reports to Codecov using tokenless OIDC, keeping coverage trends separated by package via flags.
Changes:
- Add
test:coverageTurbo pipeline + rootpnpm test:coverageentrypoint. - Configure Vitest v8 coverage output (
coverage/lcov.info) forarkor,create-arkor,cli-internal, andstudio-app, including per-package scripts and@vitest/coverage-v8devDeps. - Add Codecov configuration (
codecov.yaml) plus a CIcoveragejob that uploads each package’s LCOV under a dedicated Codecov flag via OIDC.
Reviewed changes
Copilot reviewed 12 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| turbo.json | Adds test:coverage task with coverage/** outputs for caching/artifacts. |
| package.json | Adds root-level test:coverage script that runs the Turbo task. |
| packages/arkor/vitest.config.ts | Enables v8 coverage + LCOV output for arkor. |
| packages/arkor/package.json | Adds test:coverage script and v8 coverage provider devDep. |
| packages/create-arkor/vitest.config.ts | Enables v8 coverage + LCOV output for create-arkor. |
| packages/create-arkor/package.json | Adds test:coverage script and v8 coverage provider devDep (pass with no tests). |
| packages/cli-internal/vitest.config.ts | Enables v8 coverage + LCOV output for cli-internal. |
| packages/cli-internal/package.json | Adds test:coverage script and v8 coverage provider devDep. |
| packages/studio-app/vitest.config.ts | Enables v8 coverage + LCOV output for studio-app with app-specific excludes. |
| packages/studio-app/package.json | Adds test:coverage script and v8 coverage provider devDep. |
| codecov.yaml | Configures informational coverage statuses, per-package flag scoping, and ignore rules. |
| .gitignore | Ignores generated coverage/ directories. |
| .github/workflows/ci.yaml | Adds a dedicated coverage job that runs coverage once (Node 24) and uploads per-package reports to Codecov via OIDC. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # is stored in this repo. `id-token: write` is required for the runner | ||
| # to mint the OIDC JWT that codecov-action exchanges for an upload | ||
| # token. See https://docs.codecov.com/docs/github-tokenless-uploads. | ||
| permissions: |
- Updated CI workflow to upload JUnit XML test results to Codecov for Test Analytics, improving test tracking and reporting. - Configured Vitest in all relevant packages to output JUnit XML files for compatibility with Codecov. - Added conditional checks to ensure uploads occur even if tests fail, maintaining coverage reporting stability.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7c18ee9a81
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # `!cancelled()` so they run even when this step fails; the job's | ||
| # final outcome still reflects the test result. | ||
| - name: Test with coverage | ||
| run: pnpm test:coverage |
There was a problem hiding this comment.
pnpm test:coverage invokes turbo run test:coverage with Turborepo’s default --continue=never, which cancels pending tasks after the first failing package test. In that failure scenario, remaining packages may never produce coverage/lcov.info or coverage/junit.xml, so the later Codecov upload steps only submit partial data despite the workflow’s intent to upload results even when tests fail. Use a continue mode (for example --continue / --continue=always) or run package coverage tasks separately to ensure all reports are generated.
Useful? React with 👍 / 👎.
- Added support for E2E coverage collection using c8 in the CI workflow, enabling detailed reporting of CLI test coverage. - Configured Codecov to upload E2E coverage and test results, enhancing visibility into test performance. - Updated .gitignore to exclude raw V8 coverage data and added a configuration file for c8 to manage coverage settings. - Enhanced package.json scripts in the E2E CLI to include coverage reporting capabilities.
There was a problem hiding this comment.
Pull request overview
Adds per-package coverage generation and Codecov uploads (including JUnit test analytics) across the monorepo, with a dedicated CI job using Codecov’s tokenless OIDC flow.
Changes:
- Add
test:coverageTurbo task + rootpnpm test:coverageentrypoint. - Configure Vitest v8 coverage + JUnit output for
arkor,create-arkor,cli-internal, andstudio-app; add c8-based coverage fore2e/cli. - Add Codecov configuration + CI job to upload coverage and test results per package/flag via OIDC.
Reviewed changes
Copilot reviewed 15 out of 17 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
turbo.json |
Adds test:coverage task with coverage/** outputs. |
package.json |
Adds root test:coverage script via Turbo. |
pnpm-lock.yaml |
Locks new coverage-related deps (@vitest/coverage-v8, c8) and transitive graph. |
packages/arkor/vitest.config.ts |
New Vitest config enabling v8 coverage + JUnit output. |
packages/arkor/package.json |
Adds test:coverage script and @vitest/coverage-v8 devDep. |
packages/create-arkor/vitest.config.ts |
New Vitest config enabling v8 coverage + JUnit output. |
packages/create-arkor/package.json |
Adds test:coverage script and @vitest/coverage-v8 devDep. |
packages/cli-internal/vitest.config.ts |
New Vitest config enabling v8 coverage + JUnit output. |
packages/cli-internal/package.json |
Adds test:coverage script and @vitest/coverage-v8 devDep. |
packages/studio-app/vitest.config.ts |
New Vitest config enabling v8 coverage + JUnit output for TS/TSX. |
packages/studio-app/package.json |
Adds test:coverage script and @vitest/coverage-v8 devDep. |
e2e/cli/vitest.config.ts |
Adds JUnit reporter output for Test Analytics. |
e2e/cli/package.json |
Adds test:coverage using c8 + pre-hook to build CLIs; adds c8 devDep. |
e2e/cli/.c8rc.json |
New c8 config to capture child-process v8 coverage and emit lcov. |
codecov.yaml |
Adds Codecov config: informational statuses, per-flag scoping, ignore rules. |
.gitignore |
Ignores coverage directories and raw v8 coverage artifacts. |
.github/workflows/ci.yaml |
Adds a dedicated coverage job that runs pnpm test:coverage and uploads coverage + JUnit to Codecov via OIDC. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # E2E coverage is collected via c8 wrapping vitest in e2e/cli; the | ||
| # spawned CLI children inherit NODE_V8_COVERAGE and their hits are | ||
| # remapped through tsdown's sourcemaps back into | ||
| # packages/{arkor,create-arkor}/src/**. | ||
| - name: Upload e2e-cli coverage | ||
| if: ${{ !cancelled() }} | ||
| uses: codecov/codecov-action@v5 | ||
| with: | ||
| use_oidc: true | ||
| files: ./e2e/cli/coverage/lcov.info | ||
| flags: e2e-cli | ||
| fail_ci_if_error: false |
| export default defineConfig({ | ||
| test: { | ||
| // `default` keeps normal CLI output; `junit` writes the XML that | ||
| // codecov/test-results-action consumes for Test Analytics. |
| export default defineConfig({ | ||
| test: { | ||
| // `default` keeps normal CLI output; `junit` writes the XML that | ||
| // codecov/test-results-action consumes for Test Analytics. |
| // `default` keeps normal CLI output; `junit` writes the XML that | ||
| // codecov/test-results-action consumes for Test Analytics. |
| export default defineConfig({ | ||
| test: { | ||
| // `default` keeps normal CLI output; `junit` writes the XML that | ||
| // codecov/test-results-action consumes for Test Analytics. |
| # required for the runner to mint the OIDC JWT that codecov-action | ||
| # exchanges for an upload token. See | ||
| # https://docs.codecov.com/docs/github-tokenless-uploads. | ||
| permissions: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f9135056fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - "packages/arkor/scripts/**" | ||
| - "packages/create-arkor/scripts/**" | ||
| - "packages/arkor/src/bin.ts" | ||
| - "packages/create-arkor/src/bin.ts" |
There was a problem hiding this comment.
The new ignore rule removes packages/create-arkor/src/bin.ts, which is currently the only file under packages/create-arkor/src. That means the create-arkor flag has no eligible files, so uploads from packages/create-arkor/coverage/lcov.info cannot contribute to coverage trends for that package. This defeats the stated per-package coverage tracking for create-arkor and leaves that flag effectively empty.
Useful? React with 👍 / 👎.
| - name: e2e-cli | ||
| paths: | ||
| - packages/arkor/src | ||
| - packages/create-arkor/src |
There was a problem hiding this comment.
The e2e-cli flag is scoped to packages/create-arkor/src, but the e2e coverage run collects data from packages/create-arkor/dist/**. Because create-arkor is built with sourcemaps disabled (sourcemap: false in its tsdown config), those dist paths cannot remap back to src, so Codecov's **/dist/** ignore will drop them. In practice, e2e uploads for create-arkor do not count toward this flag.
Useful? React with 👍 / 👎.
…sults - Updated CI workflow to include `disable_search: true` in Codecov action configurations for all relevant coverage and JUnit XML report uploads. - This change aims to streamline the reporting process and improve performance during coverage uploads.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3f4e44bb29
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| permissions: | ||
| id-token: write |
There was a problem hiding this comment.
contents: read to coverage job token
This job sets permissions with only id-token: write, and in GitHub Actions any unspecified permission is implicitly none; that removes repository read access for the default GITHUB_TOKEN. Because this same job uses actions/checkout, it can fail to fetch sources (especially in private repos or tighter org defaults), which blocks the entire coverage/test-results upload flow before tests run. Add contents: read alongside id-token: write in this job.
Useful? React with 👍 / 👎.
- Pass --continue to turbo's test:coverage invocation in CI so a single failing package no longer cancels the rest and starves the per-package upload steps. - Mark test:coverage as cache: false in turbo.json. The previous outputs-based caching could let turbo restore prior coverage/junit artefacts from .turbo/cache and skip running tests, which would feed Codecov stale data and defeat Test Analytics flake detection. - Stop ignoring packages/create-arkor/src/bin.ts in both vitest exclude and codecov.yaml. It is the only source file under that package, so excluding it left the create-arkor flag with zero eligible files. - Gate create-arkor's tsdown sourcemap output behind the CREATE_ARKOR_BUILD_SOURCEMAP=1 env var; the published tarball stays sourcemap-free by default. e2e/cli's pretest:coverage opts in so c8 can remap dist/bin.mjs hits back to src/bin.ts (otherwise codecov's **/dist/** ignore would drop every create-arkor E2E hit). - Update vitest.config.ts comments to reference codecov-action with report_type: test_results instead of the deprecated test-results-action.
There was a problem hiding this comment.
Pull request overview
Adds first-class per-package coverage generation and Codecov uploads across the monorepo, enabling coverage + test analytics reporting in CI using Codecov’s tokenless OIDC flow.
Changes:
- Introduce
test:coverageTurbo task + package scripts and Vitest v8 coverage config to emitcoverage/lcov.info+coverage/junit.xmlper package. - Add a dedicated CI
coveragejob that runs coverage once (Node 24) and uploads per-package Codecov flags (plus ane2e-cliflag viac8). - Add
codecov.yamlconfiguration and ignore rules; update.gitignorefor coverage artifacts.
Reviewed changes
Copilot reviewed 16 out of 18 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
turbo.json |
Adds test:coverage task wired into the Turbo graph. |
package.json |
Adds root test:coverage script. |
pnpm-lock.yaml |
Locks new coverage-related dependencies (@vitest/coverage-v8, c8, transitive deps). |
packages/arkor/vitest.config.ts |
Adds Vitest config for junit + v8 coverage output to packages/arkor/coverage/. |
packages/arkor/package.json |
Adds test:coverage script and @vitest/coverage-v8. |
packages/create-arkor/vitest.config.ts |
Adds Vitest config for junit + v8 coverage output. |
packages/create-arkor/tsdown.config.ts |
Makes sourcemaps conditional for E2E coverage remapping. |
packages/create-arkor/package.json |
Adds test:coverage and @vitest/coverage-v8. |
packages/cli-internal/vitest.config.ts |
Adds Vitest config for junit + v8 coverage output. |
packages/cli-internal/package.json |
Adds test:coverage and @vitest/coverage-v8. |
packages/studio-app/vitest.config.ts |
Adds Vitest config for junit + v8 coverage output. |
packages/studio-app/package.json |
Adds test:coverage and @vitest/coverage-v8. |
e2e/cli/vitest.config.ts |
Adds junit reporting for Codecov Test Analytics. |
e2e/cli/package.json |
Adds test:coverage via c8 + pretest:coverage build step. |
e2e/cli/.c8rc.json |
Configures c8 to collect/merge V8 coverage from spawned CLI processes. |
codecov.yaml |
Configures per-flag scoping, informational status checks, and ignore rules. |
.gitignore |
Ignores generated coverage and raw V8 coverage data directories. |
.github/workflows/ci.yaml |
Adds coverage job to run coverage + upload Codecov coverage and test results using OIDC. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # required for the runner to mint the OIDC JWT that codecov-action | ||
| # exchanges for an upload token. See | ||
| # https://docs.codecov.com/docs/github-tokenless-uploads. | ||
| permissions: |
Summary
arkor,create-arkor,cli-internal, andstudio-app, each emittinglcov.infounderpackages/<pkg>/coverage/.coverageCI job (Node 24, single run) that executespnpm test:coverageand uploads each package's report to Codecov under its own flag, so per-package trends stay separated.use_oidc: true+id-token: write) — noCODECOV_TOKENsecret is stored in this repo.codecov.yamlwith informational status checks (so coverage dips don't fail PRs while we establish a baseline), per-package flag scoping, and ignore rules for tests,dist/, scaffolding scripts, and CLI entry points.e2e/clifrom coverage on purpose: it spawns the built CLI in child processes, so v8 coverage from the parent vitest run wouldn't see source instrumentation. Can be revisited later viaNODE_V8_COVERAGE=.Test plan
pnpm installcompletes cleanly with the new@vitest/coverage-v8devDeps.pnpm test:coveragefrom the repo root producespackages/{arkor,create-arkor,cli-internal,studio-app}/coverage/lcov.info.buildmatrix still passes across all Node entries.coveragejob runs on this PR, prints Codecov upload success for each of the four flags, and the report appears at https://app.codecov.io/gh/arkorlab/arkor.Follow-ups (out of scope)
README.mdonce the first upload lands and a baseline exists.codecov.yamlthresholds (informational: false, set a realtarget) once trends stabilise.e2e/cliviaNODE_V8_COVERAGE=if E2E coverage becomes useful.