Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .c8rc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"include": ["lib/**"],
"exclude": [
"lib/log.js",
"lib/**/*.d.ts",
"test/**",
"prelude/**",
"scripts/**",
"dictionary/**"
],
"exclude-after-remap": true,
"reports-dir": "coverage",
"all": true,
"temp-directory": "coverage/tmp",
"reporter": ["text", "lcov"]
}
39 changes: 30 additions & 9 deletions .claude/rules/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,23 +9,44 @@ paths:
## Commands

```bash
yarn build # Always build first
yarn test:22 # Test with Node.js 22
yarn test:host # Test with host Node.js version
node test/test.js node22 no-npm test-50-* # Run specific test pattern
yarn test:unit # Fast in-process unit suite (node:test, ~1s, no build)
yarn test:unit:watch # Unit suite in watch mode

yarn build # Required before e2e suite
yarn test:22 # E2E with Node.js 22
yarn test:host # E2E with host Node.js version
node test/test.js node22 no-npm test-50-* # Run specific e2e pattern

yarn coverage:unit # c8 unit coverage → coverage/lcov.info
yarn coverage # Merged unit + e2e coverage (slow)
```

## Organization

- Tests live in `test/test-XX-descriptive-name/` directories (XX = execution order).
- Each test has a `main.js` entry point using utilities from `test/utils.js`.
Two suites:

- **Unit suite** (`test/unit/*.test.ts`) — `node:test` runner, imports `lib/*.ts` via `esbuild-register`. Pure in-process, no binaries produced.
- **E2E suite** (`test/test-XX-descriptive-name/main.js`) — each directory spawns the `pkg` CLI and asserts on the produced binary. XX = execution order.

Special e2e tests:

- `test-79-npm/` — npm package integration tests (only-npm).
- `test-42-fetch-all/` — verifies patches exist for all Node.js versions.

## Writing Tests

1. Create `test/test-XX-descriptive-name/` with a `main.js`
2. Use `utils.pkg.sync()` to invoke pkg
3. Verify outputs, clean up with `utils.filesAfter()`
Prefer a unit test when the thing under test is a pure function in `lib/*.ts` (parsers, path helpers, selectors, etc.) — they're ~1s and give much better iteration speed than a full build+spawn cycle.

**Unit test** (`test/unit/your-thing.test.ts`):

1. Use `import { describe, it } from 'node:test'` and `import assert from 'node:assert/strict'`.
2. Import the thing directly from `../../lib/...` — no need for `lib-es5/`.
3. For platform-specific logic, skip the wrong half with `describe(..., { skip: !onWin }, ...)`.

**E2E test** (`test/test-XX-descriptive-name/main.js`):

1. Create the directory with a `main.js`.
2. Use `utils.pkg.sync()` to invoke pkg.
3. Verify outputs, clean up with `utils.filesAfter()`.

Test artifacts (`*.exe`, `*-linux`, `*-macos`, `*-win.exe`) must be cleaned from test directories before committing.
17 changes: 13 additions & 4 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,13 @@ This workflow ensures code quality, prevents accidental commits of test artifact
The test suite is extensive and organized in numbered directories:

```bash
# Build first (required)
# Build first (required for e2e)
yarn build

# Fast in-process unit suite (node:test + esbuild-register, ~1s, no build)
yarn test:unit
yarn test:unit:watch

# Run all tests
yarn test

Expand All @@ -117,13 +121,18 @@ yarn test:host # Test with host Node.js version

# Run specific test pattern
node test/test.js node20 no-npm test-50-*

# Coverage (c8; include lib/**, merged unit + e2e into coverage/lcov.info)
yarn coverage:unit
yarn coverage:e2e
yarn coverage
```

#### Test Organization

- Tests are in `test/test-XX-*/` directories where XX indicates execution order
- Each test directory contains a `main.js` file that runs the test
- Tests use `utils.js` for common testing utilities
- `test/unit/*.test.ts` — in-process unit suite on Node's built-in `node:test` runner. Imports `lib/*.ts` directly via `esbuild-register` (no build required).
- `test/test-XX-*/main.js` — e2e suite: each directory spawns the pkg CLI and asserts on produced binaries. XX indicates execution order.
- Tests use `utils.js` for common testing utilities.
- Special tests:
- `test-79-npm/`: Tests integration with popular npm packages (only-npm)
- `test-42-fetch-all/`: Verifies patches exist for all Node.js versions
Expand Down
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,24 @@ jobs:
prelude/sea-bootstrap.bundle.js
retention-days: 1

# Coverage upload — unit-suite only on every PR (~2s). E2E coverage is
# uploaded from the canonical Ubuntu cells in `.github/workflows/test.yml`
# with `flags: e2e`; Codecov merges the two flags via carryforward (see
# codecov.yml) so PR reports reflect unit + e2e combined. The `build`
# matrix runs `yarn test:unit` on 3 OS × Node 22/24 to verify platform
# behavior; this is the single canonical unit-coverage upload.
- if: needs.changes.outputs.code == 'true'
run: yarn coverage:unit
- if: needs.changes.outputs.code == 'true'
name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
slug: yao-pkg/pkg
files: coverage/lcov.info
flags: unit
fail_ci_if_error: false

# Sanity check: `yarn build` succeeds on every supported OS + Node combo.
# Runs in parallel with tests (they don't consume this job's output) so
# failures still mark the PR red without gating the test jobs.
Expand All @@ -81,24 +99,33 @@ jobs:
run: yarn install
- if: needs.changes.outputs.code == 'true'
run: yarn build
# Unit suite is in-process (node:test + esbuild-register) and takes
# ~1s, so it rides the existing per-(os,node) build matrix. This gives
# Windows- and Node 24-specific path/runtime coverage for free — both
# matter for common.test.ts's win32 branch.
- if: needs.changes.outputs.code == 'true'
run: yarn test:unit

test_host:
needs: [changes, build_artifact]
uses: ./.github/workflows/test.yml
secrets: inherit
with:
npm_command: test:host
should_run: ${{ needs.changes.outputs.code }}

test_22:
needs: [changes, build_artifact]
uses: ./.github/workflows/test.yml
secrets: inherit
with:
npm_command: test:22
should_run: ${{ needs.changes.outputs.code }}

test_24:
needs: [changes, build_artifact]
uses: ./.github/workflows/test.yml
secrets: inherit
with:
npm_command: test:24
should_run: ${{ needs.changes.outputs.code }}
42 changes: 41 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,48 @@ jobs:
with:
name: build-output

- if: inputs.should_run == 'true'
# Two canonical Ubuntu cells (Node 22 host + test:22 target; Node 24
# host + test:24 target) wrap their e2e run in c8 to capture coverage.
# Every other cell (Windows, macOS, mixed host/target) runs the test
# command plainly — the canonical cells already exercise the same
# lib/ code paths, so extra uploads from them would just duplicate
# data for Codecov to merge and discard.
- name: Canonical cell?
id: cov
shell: bash
if: inputs.should_run == 'true'
run: |
if [[ "${{ matrix.os }}" == "ubuntu-latest" && \
( ( "${{ inputs.npm_command }}" == "test:22" && "${{ matrix.node-version }}" == "22.x" ) || \
( "${{ inputs.npm_command }}" == "test:24" && "${{ matrix.node-version }}" == "24.x" ) ) ]]; then
echo "coverage=true" >> "$GITHUB_OUTPUT"
else
echo "coverage=false" >> "$GITHUB_OUTPUT"
fi

- if: inputs.should_run == 'true' && steps.cov.outputs.coverage != 'true'
run: yarn ${{ inputs.npm_command }}
env:
CI: true
timeout-minutes: 30

- if: inputs.should_run == 'true' && steps.cov.outputs.coverage == 'true' && inputs.npm_command == 'test:22'
run: yarn coverage:e2e
env:
CI: true
timeout-minutes: 40
- if: inputs.should_run == 'true' && steps.cov.outputs.coverage == 'true' && inputs.npm_command == 'test:24'
run: yarn coverage:e2e:24
env:
CI: true
timeout-minutes: 40

- if: inputs.should_run == 'true' && steps.cov.outputs.coverage == 'true'
name: Upload e2e coverage to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
slug: yao-pkg/pkg
files: coverage/lcov.info
flags: e2e
fail_ci_if_error: false
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
# dependencies
/node_modules

# coverage reports (c8)
/coverage

# pkg uses yarn — a root package-lock.json is always accidental
/package-lock.json

Expand Down
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,12 @@
## Quick Reference

```bash
yarn build # Build (required before testing)
yarn build # Build (required before e2e testing)
yarn lint # Check lint + formatting
yarn fix # Auto-fix lint + formatting
yarn start # Watch mode (rebuild on change)
yarn test:22 # Run tests for Node.js 22
yarn test:unit # Fast in-process unit suite (node:test, ~1s)
yarn test:22 # Run e2e tests for Node.js 22
```

> `pkg` uses **yarn** for dependency management. `npm` is only used in `docs-site/` (the VitePress docs). Do not create a root `package-lock.json`.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

<p align="center">
<a href="https://github.com/yao-pkg/pkg/actions/workflows/ci.yml"><img src="https://github.com/yao-pkg/pkg/actions/workflows/ci.yml/badge.svg" alt="Build Status" /></a>
<a href="https://codecov.io/gh/yao-pkg/pkg"><img src="https://codecov.io/gh/yao-pkg/pkg/branch/main/graph/badge.svg" alt="Coverage" /></a>
<a href="https://www.npmjs.com/package/@yao-pkg/pkg"><img src="https://img.shields.io/npm/v/@yao-pkg/pkg" alt="npm version" /></a>
<a href="https://www.npmjs.com/package/@yao-pkg/pkg"><img src="https://img.shields.io/npm/dm/@yao-pkg/pkg" alt="npm downloads" /></a>
<a href="https://github.com/yao-pkg/pkg/blob/main/LICENSE"><img src="https://img.shields.io/npm/l/@yao-pkg/pkg" alt="license" /></a>
Expand Down
48 changes: 48 additions & 0 deletions codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Codecov config — keep coverage informational, don't gate PRs on it.
# Every PR uploads BOTH flags: `unit` from build_artifact (fast), and `e2e`
# from the two canonical Ubuntu e2e cells in test.yml (Node 22 + Node 24).
# Codecov merges flags per commit, so the total reflects unit + e2e natively.
coverage:
status:
project:
default:
# Report but never fail the PR. Coverage swings naturally with the
# unit-vs-e2e split and the 24h lag on e2e refreshes.
target: auto
threshold: 5%
informational: true
patch:
default:
# New lines should be covered by the unit suite when reasonable, but
# many changes land in integration-heavy modules (walker, producer,
# packer) that stay e2e-tested. Warn, don't fail.
informational: true

comment:
layout: "reach, diff, files"
behavior: default
require_changes: true

ignore:
- "lib-es5/**"
- "test/**"
- "prelude/**"
- "dictionary/**"
- "scripts/**"
- "docs-site/**"
- "**/*.d.ts"

# carryforward is a safety net: if one of the two flag uploads fails (e.g.
# Codecov outage mid-run, an e2e cell crashes), Codecov reuses the last known
# value for that flag on the new commit. With both flags uploaded per PR the
# normal case doesn't depend on it, but keeping it set avoids a zeroed-out
# report on transient upload failures.
flags:
unit:
paths:
- lib/
carryforward: true
e2e:
paths:
- lib/
carryforward: true
38 changes: 35 additions & 3 deletions docs-site/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,21 @@ This command starts an interactive process that guides you through the release u

## Testing

Before running tests, ensure you have built the project by running:
`pkg` has two test suites:

1. **Unit suite** (`test/unit/*.test.ts`) — in-process assertions using Node's built-in [`node:test`](https://nodejs.org/api/test.html) runner. Imports `lib/*.ts` directly via `esbuild-register`, so no `yarn build` is required. Runs in ~1 second.
2. **E2E suite** (`test/test-XX-*/`) — each directory spawns the `pkg` CLI and asserts on the produced binaries. Requires a prior `yarn build`.

### Unit suite

```bash
yarn test:unit # run once
yarn test:unit:watch # re-run on change
```

### E2E suite

Before running e2e tests, ensure you have built the project by running:

```bash
yarn build
Expand All @@ -42,11 +56,29 @@ node test/test.js <target> [no-npm | only-npm | all] [<flavor>]
- `[no-npm | only-npm | all]` to specify which tests to run. `no-npm` will run tests that don't require npm, `only-npm` will run against some specific npm modules, and `all` will run all tests.
- `<flavor>` to use when you want to run only tests matching a specific pattern. Example: `node test/test.js all test-99-*`. You can also set this by using `FLAVOR` environment variable.

Each test is located inside `test` directory into a dedicated folder named following the pattern `test-XX-*`. The `XX` is a number that represents the order the tests will run.
Each e2e test is located inside `test` directory into a dedicated folder named following the pattern `test-XX-*`. The `XX` is a number that represents the order the tests will run.

When running `node test/test.js all`, based on the options, each test will be run consecutively by running `main.js` file inside the test folder.

### Example test
### Coverage

`pkg` uses [c8](https://github.com/bcoe/c8) as a thin reporter over V8's built-in coverage. `NODE_V8_COVERAGE` propagates through child processes, so e2e coverage captures the `pkg` CLI executions spawned by the harness.

```bash
# Run unit coverage alone — always deterministic; clears coverage/tmp first.
yarn coverage:unit

# Append e2e coverage on top of whatever is in coverage/tmp (uses `c8 --clean=false`).
# After a fresh `yarn coverage:unit`, this produces a merged report. Without
# a prior run it produces an e2e-only report. For a guaranteed e2e-only view,
# delete `coverage/` first. (Slow — runs the full e2e matrix.)
yarn coverage:e2e

# Full merged report: cleans, runs unit, then appends e2e, then emits lcov+text.
yarn coverage
```
Comment thread
robertsLando marked this conversation as resolved.

### Example e2e test

Create a directory named `test-XX-<name>` and inside it create a `main.js` file with the following content:

Expand Down
5 changes: 4 additions & 1 deletion eslint.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@ module.exports = [
languageOptions: {
parser: tsParser,
parserOptions: {
project: './tsconfig.json',
// tsconfig.test.json extends tsconfig.json and additionally includes
// test/unit/**/*.test.ts so both lib and unit-suite sources resolve
// to a configured project for typescript-eslint.
project: ['./tsconfig.json', './tsconfig.test.json'],
ecmaVersion: 'latest',
sourceType: 'module',
},
Expand Down
Loading
Loading