Skip to content

Glossary

Jump to bottom
Matěj Chalk edited this page Sep 12, 2023 · 2 revisions

Entity-relationship diagram:

QM Data Model drawio

Core config

User-facing configuration for CLI tool.

Consists of:

Plugin

Built-in or custom analyzer for some open-source tool.

A plugin accepts it's own configuration and provides a runner command, which produces output in a JSON format supported by the core config (see audits, issues, groups).

Examples:

  • ESLint
  • Stylelint
  • TypeScript
  • Lighthouse CI
  • user-flow
  • bundle size
  • test coverage
  • NPM audit
  • test results

Audit

A measurable metric provided by a plugin.

An audit result is composed of:

  • a score (0-1),
  • a numeric value (e.g. count, duration, size),
  • optional details, including a list of issues.

Examples:

  • ESLint rule (e.g. no-unused-vars)
  • TypeScript compiler flag (e.g. strictNullChecks)
  • Lighthouse audit (e.g. first-contentful-paint)
  • bundle (e.g. main)
  • Jest coverage types (e.g. branches or statements)
  • NPM dependency group (e.g. prod or dev)

Issue

A specific occurrence within a given audit result of some error, warning, vulnerability, test result, etc. Consists of a message, severity and possibly additional context data (e.g. location in source code, screenshot, NPM package).

Examples:

  • ESLint error or warning
  • TypeScript compiler error
  • Lighthouse audit details item
  • uncovered lines in test coverage report
  • vulnerability from NPM audit
  • passed or failed E2E test

Category

Serves to aggregate various audits into a single score. The score is calculated by assigning weights to selected audit or group scores (weight 0 may be used to merely display output as additional info).

Unlike a group, a category is part of the core config and thus may be composed from multiple other plugins.

Examples:

  • "performance" - made up of performance-related audits from Lighthouse, ESLint, bundle-size and other plugins
  • "code style" - made up of audits from ESLint and Prettier plugins
  • "bugs" - made up of audits from some bug tracker and test results plugins

Group

Aggregates audits into a single score just like a category does, but is defined by a specific plugin, not in the core config. Intended to simplify referencing audits in category or budget configurations.

Not to be confused with "group" in Lighthouse CI terminology, which just defines a section where to display results.

Examples:

  • default Lighthouse category (e.g. performance or accessibility)

Report

The output of running CLI with a given config for a given commit. Contains metadata and results for all plugins, categories, audits, etc.

Budget

Defines assertion rules in the core config, which may cause the analysis to fail if certain thresholds aren't met. References audits, groups or categories, and defines some limit for each - either as a minimum score, maximum numeric value, or maximum number of issues (of some minimum severity).

Examples:

  • accessibility score must not dip below 0.9
  • main bundle cannot exceed 3 MB
  • no critical- or high-severity vulnerabilities in production depedencies
Clone this wiki locally