Skip to content

Commit

Permalink
feat!: add @vitest/coverage-v8 package
Browse files Browse the repository at this point in the history
  • Loading branch information
AriPerkkio committed May 16, 2023
1 parent 4a807df commit c22898b
Show file tree
Hide file tree
Showing 24 changed files with 4,050 additions and 97 deletions.
2 changes: 1 addition & 1 deletion docs/.vitepress/components/FeaturesList.vue
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
<ListItem><a target="_blank" href="https://www.chaijs.com/" rel="noopener noreferrer">Chai</a> built-in for assertions + <a target="_blank" href="https://jestjs.io/docs/expect" rel="noopener noreferrer">Jest expect</a> compatible APIs</ListItem>
<ListItem><a target="_blank" href="https://github.com/tinylibs/tinyspy" rel="noopener noreferrer">Tinyspy</a> built-in for mocking</ListItem>
<ListItem><a target="_blank" href="https://github.com/capricorn86/happy-dom" rel="noopener noreferrer">happy-dom</a> or <a target="_blank" href="https://github.com/jsdom/jsdom" rel="noopener noreferrer">jsdom</a> for DOM mocking</ListItem>
<ListItem>Code coverage via <a target="_blank" href="https://github.com/bcoe/c8" rel="noopener noreferrer">c8</a> or <a target="_blank" href="https://istanbul.js.org/" rel="noopener noreferrer">istanbul</a></ListItem>
<ListItem>Code coverage via <a target="_blank" href="https://v8.dev/blog/javascript-code-coverage" rel="noopener noreferrer">v8</a> or <a target="_blank" href="https://istanbul.js.org/" rel="noopener noreferrer">istanbul</a></ListItem>
<ListItem>Rust-like <a href="/guide/in-source">in-source testing</a></ListItem>
<ListItem>Type Testing via <a target="_blank" href="https://github.com/mmkal/expect-type" rel="noopener noreferrer">expect-type</a></ListItem>
</ul>
Expand Down
2 changes: 1 addition & 1 deletion docs/advanced/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ await vitest?.close()
`startVitest` function returns `Vitest` instance if tests can be started. It returns `undefined`, if one of the following occurs:
- Vitest didn't find "vite" package (usually installed with Vitest)
- If coverage is enabled and run mode is "test", but the coverage package is not installed (`@vitest/coverage-c8` or `@vitest/coverage-istanbul`)
- If coverage is enabled and run mode is "test", but the coverage package is not installed (`@vitest/coverage-v8` or `@vitest/coverage-istanbul`)
- If the environment package is not installed (`jsdom`/`happy-dom`/`@edge-runtime/vm`)
If `undefined` is returned or tests failed during the run, Vitest sets `process.exitCode` to `1`.
Expand Down
44 changes: 24 additions & 20 deletions docs/config/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -645,7 +645,11 @@ Isolate environment for each test file. Does not work if you disable [`--threads

### coverage<NonProjectOption />

You can use [`c8`](https://github.com/bcoe/c8), [`istanbul`](https://istanbul.js.org/) or [a custom coverage solution](/guide/coverage#custom-coverage-provider) for coverage collection.
You can use [`c8`](https://github.com/bcoe/c8), [`v8`](https://v8.dev/blog/javascript-code-coverage), [`istanbul`](https://istanbul.js.org/) or [a custom coverage solution](/guide/coverage#custom-coverage-provider) for coverage collection.

::: info
The `c8` provider is being replaced by `v8` and will eventually be removed completely.
:::

You can provide coverage options to CLI with dot notation:

Expand All @@ -659,7 +663,7 @@ If you are using coverage options with dot notation, don't forget to specify `--

#### coverage.provider

- **Type:** `'c8' | 'istanbul' | 'custom'`
- **Type:** `'c8' | 'v8' | 'istanbul' | 'custom'`
- **Default:** `'c8'`
- **CLI:** `--coverage.provider=<provider>`

Expand All @@ -669,7 +673,7 @@ Use `provider` to select the tool for coverage collection.

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.enabled`, `--coverage.enabled=false`

Enables coverage collection. Can be overridden using `--coverage` CLI option.
Expand All @@ -678,7 +682,7 @@ Enables coverage collection. Can be overridden using `--coverage` CLI option.

- **Type:** `string[]`
- **Default:** `['**']`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.include=<path>`, `--coverage.include=<path1> --coverage.include=<path2>`

List of files included in coverage as glob patterns
Expand All @@ -687,7 +691,7 @@ List of files included in coverage as glob patterns

- **Type:** `string | string[]`
- **Default:** `['.js', '.cjs', '.mjs', '.ts', '.mts', '.cts', '.tsx', '.jsx', '.vue', '.svelte']`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.extension=<extension>`, `--coverage.extension=<extension1> --coverage.extension=<extension2>`

#### coverage.exclude
Expand All @@ -710,7 +714,7 @@ List of files included in coverage as glob patterns
'**/.{eslint,mocha,prettier}rc.{js,cjs,yml}',
]
```
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.exclude=<path>`, `--coverage.exclude=<path1> --coverage.exclude=<path2>`

List of files excluded from coverage as glob patterns.
Expand All @@ -719,7 +723,7 @@ List of files excluded from coverage as glob patterns.

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.all`, `--coverage.all=false`

Whether to include all files, including the untested ones into report.
Expand All @@ -728,7 +732,7 @@ Whether to include all files, including the untested ones into report.

- **Type:** `boolean`
- **Default:** `true`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.clean`, `--coverage.clean=false`

Clean coverage results before running tests
Expand All @@ -737,7 +741,7 @@ Clean coverage results before running tests

- **Type:** `boolean`
- **Default:** `true`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.cleanOnRerun`, `--coverage.cleanOnRerun=false`

Clean coverage report on watch rerun
Expand All @@ -746,7 +750,7 @@ Clean coverage report on watch rerun

- **Type:** `string`
- **Default:** `'./coverage'`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.reportsDirectory=<path>`

Directory to write coverage report to.
Expand All @@ -755,7 +759,7 @@ Directory to write coverage report to.

- **Type:** `string | string[] | [string, {}][]`
- **Default:** `['text', 'html', 'clover', 'json']`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.reporter=<reporter>`, `--coverage.reporter=<reporter1> --coverage.reporter=<reporter2>`

Coverage reporters to use. See [istanbul documentation](https://istanbul.js.org/docs/advanced/alternative-reporters/) for detailed list of all reporters. See [`@types/istanbul-reporter`](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/276d95e4304b3670eaf6e8e5a7ea9e265a14e338/types/istanbul-reports/index.d.ts) for details about reporter specific options.
Expand All @@ -782,7 +786,7 @@ Since Vitest 0.31.0, you can check your coverage report in Vitest UI: check [Vit

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.skipFull`, `--coverage.skipFull=false`

Do not show files with 100% statement, branch, and function coverage.
Expand All @@ -791,7 +795,7 @@ Do not show files with 100% statement, branch, and function coverage.

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.perFile`, `--coverage.perFile=false`

Check thresholds per file.
Expand All @@ -801,7 +805,7 @@ See `lines`, `functions`, `branches` and `statements` for the actual thresholds.

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.thresholdAutoUpdate=<boolean>`

Update threshold values `lines`, `functions`, `branches` and `statements` to configuration file when current coverage is above the configured thresholds.
Expand All @@ -810,7 +814,7 @@ This option helps to maintain thresholds when coverage is improved.
#### coverage.lines

- **Type:** `number`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.lines=<number>`

Threshold for lines.
Expand All @@ -819,7 +823,7 @@ See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-threshol
#### coverage.functions

- **Type:** `number`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.functions=<number>`

Threshold for functions.
Expand All @@ -828,7 +832,7 @@ See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-threshol
#### coverage.branches

- **Type:** `number`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.branches=<number>`

Threshold for branches.
Expand All @@ -837,7 +841,7 @@ See [istanbul documentation](https://github.com/istanbuljs/nyc#coverage-threshol
#### coverage.statements

- **Type:** `number`
- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`
- **CLI:** `--coverage.statements=<number>`

Threshold for statements.
Expand Down Expand Up @@ -874,7 +878,7 @@ Specifies the directories that are used when `--all` is enabled.

- **Type:** `boolean`
- **Default:** `false`
- **Available for providers:** `'c8'`
- **Available for providers:** `'c8' | 'v8'`
- **CLI:** `--coverage.100`, `--coverage.100=false`

Shortcut for `--check-coverage --lines 100 --functions 100 --branches 100 --statements 100`.
Expand Down Expand Up @@ -913,7 +917,7 @@ See [istanbul documentation](https://github.com/istanbuljs/nyc#ignoring-methods)
}
```

- **Available for providers:** `'c8' | 'istanbul'`
- **Available for providers:** `'c8' | 'v8' | 'istanbul'`

Watermarks for statements, lines, branches and functions. See [istanbul documentation](https://github.com/istanbuljs/nyc#high-and-low-watermarks) for more information.

Expand Down
18 changes: 11 additions & 7 deletions docs/guide/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,19 @@ title: Coverage | Guide

Vitest supports Native code coverage via [`c8`](https://github.com/bcoe/c8) and instrumented code coverage via [`istanbul`](https://istanbul.js.org/).

:::info
The `c8` coverage provider is being replaced by [`v8`](https://v8.dev/blog/javascript-code-coverage) and will eventually be removed completely.
:::

## Coverage Providers

:::tip
Since Vitest v0.22.0
:::

Both `c8` and `istanbul` support are optional. By default, `c8` will be used.
Both `v8` and `istanbul` support are optional. By default, `c8` will be used.

You can select the coverage tool by setting `test.coverage.provider` to either `c8` or `istanbul`:
You can select the coverage tool by setting `test.coverage.provider` to `v8` or `istanbul`:

```ts
// vite.config.ts
Expand All @@ -23,7 +27,7 @@ import { defineConfig } from 'vitest/config'
export default defineConfig({
test: {
coverage: {
provider: 'istanbul' // or 'c8'
provider: 'istanbul' // or 'v8'
},
},
})
Expand All @@ -34,8 +38,8 @@ When you start the Vitest process, it will prompt you to install the correspondi
Or if you prefer to install them manually:

```bash
# For c8
npm i -D @vitest/coverage-c8
# For v8
npm i -D @vitest/coverage-v8

# For istanbul
npm i -D @vitest/coverage-istanbul
Expand Down Expand Up @@ -138,7 +142,7 @@ export default defineConfig({

Both coverage providers have their own ways how to ignore code from coverage reports:

- [`c8`](https://github.com/bcoe/c8#ignoring-uncovered-lines-functions-and-blocks)
- [`v8`](https://github.com/istanbuljs/v8-to-istanbul#ignoring-uncovered-lines)
- [`ìstanbul`](https://github.com/istanbuljs/nyc#parsing-hints-ignoring-lines)

When using TypeScript the source codes are transpiled using `esbuild`, which strips all comments from the source codes ([esbuild#516](https://github.com/evanw/esbuild/issues/516)).
Expand All @@ -153,7 +157,7 @@ Beware that these ignore hints may now be included in final production build as
if (condition) {
```

For `c8` this does not cause any issues. You can use `c8 ignore` comments with Typescript as usual:
For `v8` this does not cause any issues. You can use `c8 ignore` comments with Typescript as usual:

<!-- eslint-skip -->
```ts
Expand Down
2 changes: 1 addition & 1 deletion docs/guide/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ Learn more at [Mocking](/guide/mocking).

## Coverage

Vitest supports Native code coverage via [`c8`](https://github.com/bcoe/c8) and instrumented code coverage via [`istanbul`](https://istanbul.js.org/).
Vitest supports Native code coverage via [`v8`](https://v8.dev/blog/javascript-code-coverage) and instrumented code coverage via [`istanbul`](https://istanbul.js.org/).

```json
{
Expand Down
9 changes: 9 additions & 0 deletions packages/coverage-c8/src/provider.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,15 @@ export class C8CoverageProvider extends BaseCoverageProvider implements Coverage
branches: config['100'] ? 100 : config.branches,
statements: config['100'] ? 100 : config.statements,
}

const banner = ' DEPRECATION '
this.ctx.logger.log(
c.bgYellow(c.black((banner))),
c.yellow('@vitest/coverage-c8 is being replaced by @vitest/coverage-v8.'),
c.yellow(`\n${' '.repeat(banner.length)} See`),
c.blue(c.underline('https://github.com/vitest-dev/vitest/pull/3339')),
c.yellow('for more information.'),
)
}

resolveOptions() {
Expand Down
2 changes: 1 addition & 1 deletion packages/coverage-istanbul/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
"prepublishOnly": "pnpm build"
},
"peerDependencies": {
"vitest": ">=0.30.0 <1"
"vitest": ">=0.32.0 <1"
},
"dependencies": {
"istanbul-lib-coverage": "^3.2.0",
Expand Down
64 changes: 10 additions & 54 deletions packages/coverage-istanbul/src/provider.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { existsSync, promises as fs } from 'node:fs'
import { relative, resolve } from 'pathe'
import { resolve } from 'pathe'
import type { AfterSuiteRunMeta, CoverageIstanbulOptions, CoverageProvider, ReportContext, ResolvedCoverageOptions, Vitest } from 'vitest'
import { coverageConfigDefaults, defaultExclude, defaultInclude } from 'vitest/config'
import { BaseCoverageProvider } from 'vitest/coverage'
Expand All @@ -16,8 +16,6 @@ import { COVERAGE_STORE_KEY } from './constants'

type Options = ResolvedCoverageOptions<'istanbul'>

type Threshold = 'lines' | 'functions' | 'statements' | 'branches'

interface TestExclude {
new(opts: {
cwd?: string | string[]
Expand Down Expand Up @@ -146,11 +144,15 @@ export class IstanbulCoverageProvider extends BaseCoverageProvider implements Co
|| this.options.functions
|| this.options.lines
|| this.options.statements) {
this.checkThresholds(coverageMap, {
branches: this.options.branches,
functions: this.options.functions,
lines: this.options.lines,
statements: this.options.statements,
this.checkThresholds({
coverageMap,
thresholds: {
branches: this.options.branches,
functions: this.options.functions,
lines: this.options.lines,
statements: this.options.statements,
},
perFile: this.options.perFile,
})
}

Expand All @@ -169,52 +171,6 @@ export class IstanbulCoverageProvider extends BaseCoverageProvider implements Co
}
}

checkThresholds(coverageMap: CoverageMap, thresholds: Record<Threshold, number | undefined>) {
// Construct list of coverage summaries where thresholds are compared against
const summaries = this.options.perFile
? coverageMap.files()
.map((file: string) => ({
file,
summary: coverageMap.fileCoverageFor(file).toSummary(),
}))
: [{
file: null,
summary: coverageMap.getCoverageSummary(),
}]

// Check thresholds of each summary
for (const { summary, file } of summaries) {
for (const thresholdKey of ['lines', 'functions', 'statements', 'branches'] as const) {
const threshold = thresholds[thresholdKey]

if (threshold !== undefined) {
const coverage = summary.data[thresholdKey].pct

if (coverage < threshold) {
process.exitCode = 1

/*
* Generate error message based on perFile flag:
* - ERROR: Coverage for statements (33.33%) does not meet threshold (85%) for src/math.ts
* - ERROR: Coverage for statements (50%) does not meet global threshold (85%)
*/
let errorMessage = `ERROR: Coverage for ${thresholdKey} (${coverage}%) does not meet`

if (!this.options.perFile)
errorMessage += ' global'

errorMessage += ` threshold (${threshold}%)`

if (this.options.perFile && file)
errorMessage += ` for ${relative('./', file).replace(/\\/g, '/')}`

console.error(errorMessage)
}
}
}
}
}

async includeUntestedFiles(coverageMap: CoverageMap) {
// Load, instrument and collect empty coverages from all files which
// are not already in the coverage map
Expand Down

0 comments on commit c22898b

Please sign in to comment.