Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests(ci): run unit tests on all active node versions #12513

Merged
merged 2 commits into from
May 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 1 addition & 65 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: 馃挕馃彔
name: CI

on:
push:
Expand Down Expand Up @@ -71,67 +71,3 @@ jobs:
with:
name: dist
path: dist/

# `unit` includes just unit and proto tests.
unit:
strategy:
matrix:
os: [ubuntu-latest, windows-latest]
runs-on: ${{ matrix.os }}
name: unit - ${{ matrix.os == 'ubuntu-latest' && 'Ubuntu' || 'Windows' }}

steps:
- name: git clone
uses: actions/checkout@v2
with:
# Depth of at least 2 for codecov coverage diffs. See https://github.com/GoogleChrome/lighthouse/pull/12079
fetch-depth: 2

- name: Use Node.js 12.x
uses: actions/setup-node@v1
with:
node-version: 12.x

- name: Set up protoc
uses: arduino/setup-protoc@64c0c85d18e984422218383b81c52f8b077404d3
with:
version: '3.7.1'
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 2.7
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install protobuf==3.7.1
- run: yarn install --frozen-lockfile --network-timeout 1000000

- run: yarn test-proto # Run before unit-core because the roundtrip json is needed for proto tests.

- run: sudo apt-get install xvfb
if: matrix.os == 'ubuntu-latest'

- name: yarn unit
run: |
xvfb-run --auto-servernum yarn unit:cicoverage
yarn c8 report --reporter text-lcov > unit-coverage.lcov
if: matrix.os == 'ubuntu-latest'
- name: Upload test coverage to Codecov
if: matrix.os == 'ubuntu-latest'
uses: codecov/codecov-action@6004246f47ab62d32be025ce173b241cd84ac58e
with:
flags: unit
file: ./unit-coverage.lcov

# For windows, just test the potentially platform-specific code.
- name: yarn unit-cli
run: yarn unit-cli
if: matrix.os == 'windows-latest'
- run: yarn diff:sample-json
if: matrix.os == 'windows-latest'

# Fail if any changes were written to any source files or generated untracked files (ex, from -GA).
- run: git add -A && git diff --cached --exit-code
90 changes: 90 additions & 0 deletions .github/workflows/unit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
name: unit

on:
push:
branches: [master]
pull_request: # run on all PRs, not just PRs to a particular branch

jobs:
# `unit` includes just unit and proto tests.
unit:
strategy:
matrix:
node: ['12', '14', '16']
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we just run on ['12', '16']? Not sure if there's a lot of downside to running on all three besides more jobs queued up and more chances for flakes

runs-on: ubuntu-latest
name: node ${{ matrix.node }}
env:
LATEST_NODE: '16'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT it's not possible to refer to the matrix.node full array in steps nor env variables inside matrix (even with fancy fromJSON stuff), so this was the best I could do to make a way to keep the conditionals on latest node easy to keep up to date.


steps:
- name: git clone
uses: actions/checkout@v2
with:
# Depth of at least 2 for codecov coverage diffs. See https://github.com/GoogleChrome/lighthouse/pull/12079
fetch-depth: 2

- name: Use Node.js ${{ matrix.node }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node }}

- name: Set up protoc
uses: arduino/setup-protoc@64c0c85d18e984422218383b81c52f8b077404d3
with:
version: '3.7.1'
repo-token: ${{ secrets.GITHUB_TOKEN }}

- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 2.7
- name: Install Python dependencies
run: |
python -m pip install --upgrade pip
pip install protobuf==3.7.1

- run: yarn install --frozen-lockfile --network-timeout 1000000

- run: yarn test-proto # Run before unit-core because the roundtrip json is needed for proto tests.

- run: sudo apt-get install xvfb

- name: yarn unit
if: ${{ matrix.node != env.LATEST_NODE }}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only running coverage on latest node means better coverage (since it'll have the latest v8 coverage updates) and saves ~3 minutes on the unit run for those not doing coverage

run: xvfb-run --auto-servernum yarn unit:ci

# Only gather coverage on latest node, where c8 is the most accurate.
- name: yarn unit:coverage
if: ${{ matrix.node == env.LATEST_NODE }}
run: |
xvfb-run --auto-servernum yarn unit:cicoverage
yarn c8 report --reporter text-lcov > unit-coverage.lcov
- name: Upload test coverage to Codecov
if: ${{ matrix.node == env.LATEST_NODE }}
uses: codecov/codecov-action@6004246f47ab62d32be025ce173b241cd84ac58e
with:
flags: unit
file: ./unit-coverage.lcov

# For windows, just test the potentially platform-specific code.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the existing unit-windows is different enough from the full unit jobs that it's worth just keeping it separate instead of another dimension in the matrix

unit-windows:
runs-on: windows-latest
name: Windows

steps:
- name: git clone
uses: actions/checkout@v2

- name: Use Node.js 12.x
uses: actions/setup-node@v1
with:
node-version: 12.x

- run: yarn install --frozen-lockfile --network-timeout 1000000

- name: yarn unit-cli
run: yarn unit-cli
- run: yarn diff:sample-json

# Fail if any changes were written to any source files or generated untracked files (ex, from -GA).
- run: git add -A && git diff --cached --exit-code
1 change: 1 addition & 0 deletions lighthouse-core/test/lib/i18n/locales-test.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ describe('locales', () => {
in: 'id',
iw: 'he',
mo: 'ro',
tl: 'fil',
connorjclark marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in node 16, the updated ICU version gives 'fil' for Intl.getCanonicalLocales('tl') (as anticipated), so need to add it here.

};

for (const locale of Object.keys(locales)) {
Expand Down
2 changes: 1 addition & 1 deletion readme.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Lighthouse [![GitHub Actions Status Badge](https://github.com/GoogleChrome/lighthouse/workflows/馃挕馃彔/badge.svg)](https://github.com/GoogleChrome/lighthouse/actions/workflows/ci.yml) [![GitHub Actions Status Badge](https://github.com/GoogleChrome/lighthouse/workflows/smoke/badge.svg)](https://github.com/GoogleChrome/lighthouse/actions/workflows/smoke.yml) [![Coverage Status](https://codecov.io/gh/GoogleChrome/lighthouse/branch/master/graph/badge.svg)](https://codecov.io/gh/GoogleChrome/lighthouse) [![Build tracker for Lighthouse](https://img.shields.io/badge/buildtracker-ok-blue)](https://lh-build-tracker.herokuapp.com/) [![NPM lighthouse package](https://img.shields.io/npm/v/lighthouse.svg)](https://npmjs.org/package/lighthouse)
# Lighthouse [![GitHub Actions Status Badge](https://github.com/GoogleChrome/lighthouse/workflows/CI/badge.svg)](https://github.com/GoogleChrome/lighthouse/actions/workflows/ci.yml) [![GitHub Actions Status Badge](https://github.com/GoogleChrome/lighthouse/workflows/unit/badge.svg)](https://github.com/GoogleChrome/lighthouse/actions/workflows/unit.yml) [![GitHub Actions Status Badge](https://github.com/GoogleChrome/lighthouse/workflows/smoke/badge.svg)](https://github.com/GoogleChrome/lighthouse/actions/workflows/smoke.yml) [![Coverage Status](https://codecov.io/gh/GoogleChrome/lighthouse/branch/master/graph/badge.svg)](https://codecov.io/gh/GoogleChrome/lighthouse) [![Build tracker for Lighthouse](https://img.shields.io/badge/buildtracker-ok-blue)](https://lh-build-tracker.herokuapp.com/) [![NPM lighthouse package](https://img.shields.io/npm/v/lighthouse.svg)](https://npmjs.org/package/lighthouse)

> Lighthouse analyzes web apps and web pages, collecting modern performance metrics and insights on developer best practices.
Expand Down