Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactor utils #9053

Merged
merged 1 commit into from
Oct 6, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/pre-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
with:
ssh-key: ${{ secrets.SSH_PRIVATE_KEY }}
- name: Pin dependencies
run: utils/pin_dependencies.py
run: tools/pin_dependencies.py
- name: Install dependencies
run: npm install
- name: Build
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/tot-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
PUPPETEER_PRODUCT=firefox npm install
npm install
ls .local-chromium .local-firefox
REV=$(node utils/check_availability.js -p linux)
REV=$(node tools/check_availability.js -p linux)
echo "Installing revision $REV"
cat src/revisions.ts | sed "s/[0-9]\{6,\}/$REV/" > src/revisions.ts.replaced
mv src/revisions.ts.replaced src/revisions.ts
Expand Down Expand Up @@ -77,7 +77,7 @@ jobs:
PUPPETEER_PRODUCT=firefox npm install
npm install
ls .local-chromium .local-firefox
REV=$(node utils/check_availability.js -p linux)
REV=$(node tools/check_availability.js -p linux)
echo "Installing revision $REV"
cat src/revisions.ts | sed "s/[0-9]\{6,\}/$REV/" > src/revisions.ts.replaced
mv src/revisions.ts.replaced src/revisions.ts
Expand Down Expand Up @@ -115,7 +115,7 @@ jobs:
PUPPETEER_PRODUCT=firefox npm install
npm install
ls .local-chromium .local-firefox
REV=$(node utils/check_availability.js -p linux)
REV=$(node tools/check_availability.js -p linux)
echo "Installing revision $REV"
cat src/revisions.ts | sed "s/[0-9]\{6,\}/$REV/" > src/revisions.ts.replaced
mv src/revisions.ts.replaced src/revisions.ts
Expand Down
16 changes: 8 additions & 8 deletions docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ We also have other tests such as `test:types` that tests types and

Puppeteer is using a custom test runner on top of Mocha that consults
the [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
to see if a given test result is expected or not. See more info about the test runner in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
to see if a given test result is expected or not. See more info about the test runner in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).

## Code reviews

Expand Down Expand Up @@ -97,8 +97,8 @@ The following is a description of the primary folders in Puppeteer:
- `src` - contains the source code for Puppeteer.
- `test/src` - contains the source code for Puppeteer tests.
- `utils`/`scripts` - contains various scripts.
- `utils/testserver` - contains the source code for our test servers in testing.
- `utils/mochaRunner` - contains the source code for our test runner.
- `tools/testserver` - contains the source code for our test servers in testing.
- `tools/mochaRunner` - contains the source code for our test runner.
- `compat` - contains code separated by module import type. See [`compat/README.md`](https://github.com/puppeteer/puppeteer/blob/main/compat/README.md) for details.
- `test-d` contains type tests using [`tsd`](https://github.com/SamVerschueren/tsd).
- `third_party` contains all dependencies that we vendor into the final build. See the [`third_party/README.md`](https://github.com/puppeteer/puppeteer/blob/main/third_party/README.md) for details.
Expand Down Expand Up @@ -251,7 +251,7 @@ BINARY=<path-to-executable> npm run test:chrome:headless # Or npm run test:firef
```

If a test is expected to fail on certain configurations or became flaky, update [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
to reflect that. See more info about TestExpectations.json in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
to reflect that. See more info about TestExpectations.json in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).

## API Coverage

Expand All @@ -269,7 +269,7 @@ The following steps are needed to update the Chromium version.

1. Find a suitable Chromium revision
Not all revisions have builds for all platforms, so we need to find one that does.
To do so, run `utils/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `utils/check_availability.js -help` for more options).
To do so, run `tools/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `tools/check_availability.js -help` for more options).
1. Update `src/revisions.ts` with the found revision number.
1. Update `versions.js` with the new Chromium-to-Puppeteer version mapping and update `lastMaintainedChromiumVersion` with the latest stable Chrome version.
1. Run `npm run check:protocol-revision`.
Expand All @@ -281,11 +281,11 @@ The following steps are needed to update the Chromium version.

### Bisecting upstream changes

Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `utils/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:
Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `tools/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:

```sh
node utils/bisect.js --good 686378 --bad 706915 script.js
node utils/bisect.js --unit-test Response.fromCache
node tools/bisect.js --good 686378 --bad 706915 script.js
node tools/bisect.js --unit-test Response.fromCache
```

By default, it will use the Chromium revision in `src/revisions.ts` from the `main` branch and from the working tree to determine the range to bisect.
Expand Down
6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"scripts": {
"build:test": "wireit",
"build": "npm run build --workspaces --if-present",
"check:pinned-deps": "tsx scripts/ensure-pinned-deps",
"check:pinned-deps": "tsx tools/ensure-pinned-deps",
"check": "npm run check --workspaces --if-present && run-p check:*",
"clean": "npm run clean --workspaces --if-present && rimraf **/.wireit",
"commitlint": "commitlint --from=HEAD~1",
Expand All @@ -17,7 +17,7 @@
"format:eslint": "eslint --ext js --ext ts --fix .",
"format:prettier": "prettier --write .",
"format": "run-s format:*",
"generate:markdown": "tsx utils/generate_docs.ts",
"generate:markdown": "tsx tools/generate_docs.ts",
"lint:eslint": "([ \"$CI\" = true ] && eslint --ext js --ext ts --quiet -f codeframe . || eslint --ext js --ext ts .)",
"lint:prettier": "prettier --check .",
"lint": "run-s lint:prettier lint:eslint",
Expand All @@ -32,7 +32,7 @@
"test:firefox": "run-s test:firefox:*",
"test:install": "test/test-install.sh",
"test:types": "tsd -t packages/puppeteer",
"test": "cross-env PUPPETEER_DEFERRED_PROMISE_DEBUG_TIMEOUT=20000 node utils/mochaRunner/lib/main.js"
"test": "cross-env PUPPETEER_DEFERRED_PROMISE_DEBUG_TIMEOUT=20000 node tools/mochaRunner/lib/main.js"
},
"wireit": {
"build:test": {
Expand Down
2 changes: 1 addition & 1 deletion packages/puppeteer-core/src/injected/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

This folder contains code that is injected into every Puppeteer execution context. Each file is transpiled using esbuild into a script in `src/generated` which is then imported into server code.

See `utils/generate_injected.ts` for more information.
See `tools/generate_injected.ts` for more information.
2 changes: 1 addition & 1 deletion packages/puppeteer-core/tools/generate_sources.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import {mkdir, mkdtemp, readFile, rm, writeFile} from 'fs/promises';
import {sync as glob} from 'glob';
import path, {join, resolve} from 'path';
import {chdir} from 'process';
import {job} from '../../../utils/internal/job.js';
import {job} from '../../../tools/internal/job.js';

const packageRoot = resolve(join(__dirname, '..'));
chdir(packageRoot);
Expand Down
2 changes: 1 addition & 1 deletion test/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@
"rootDir": "src"
},
"include": ["src"],
"references": [{"path": "../utils/mochaRunner/tsconfig.json"}]
"references": [{"path": "../tools/mochaRunner/tsconfig.json"}]
}
6 changes: 3 additions & 3 deletions utils/bisect.js → tools/bisect.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ Parameters:
--script path to a script that returns non-zero code for BAD revisions and 0 for good

Example:
node utils/bisect.js --unit-test test
node utils/bisect.js --good 577361 --bad 599821 --script simple.js
node utils/bisect.js --good 577361 --bad 599821 --unit-test test
node tools/bisect.js --unit-test test
node tools/bisect.js --good 577361 --bad 599821 --script simple.js
node tools/bisect.js --good 577361 --bad 599821 --unit-test test
`;

if (argv.h || argv.help) {
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Mocha Runner is a test runner on top of mocha. It uses `/test/TestSuites.json` a
## Running tests for Mocha Runner itself.

```
npm run build:test && npx c8 node utils/mochaRunner/lib/test.js
npm run build:test && npx c8 node tools/mochaRunner/lib/test.js
```

## Running tests using Mocha Runner
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
12 changes: 0 additions & 12 deletions utils/tsconfig.json

This file was deleted.

16 changes: 8 additions & 8 deletions website/versioned_docs/version-18.2.0/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ We also have other tests such as `test:types` that tests types and

Puppeteer is using a custom test runner on top of Mocha that consults
the [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
to see if a given test result is expected or not. See more info about the test runner in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
to see if a given test result is expected or not. See more info about the test runner in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).

## Code reviews

Expand Down Expand Up @@ -97,8 +97,8 @@ The following is a description of the primary folders in Puppeteer:
- `src` - contains the source code for Puppeteer.
- `test/src` - contains the source code for Puppeteer tests.
- `utils`/`scripts` - contains various scripts.
- `utils/testserver` - contains the source code for our test servers in testing.
- `utils/mochaRunner` - contains the source code for our test runner.
- `tools/testserver` - contains the source code for our test servers in testing.
- `tools/mochaRunner` - contains the source code for our test runner.
- `compat` - contains code separated by module import type. See [`compat/README.md`](https://github.com/puppeteer/puppeteer/blob/main/compat/README.md) for details.
- `test-d` contains type tests using [`tsd`](https://github.com/SamVerschueren/tsd).
- `third_party` contains all dependencies that we vendor into the final build. See the [`third_party/README.md`](https://github.com/puppeteer/puppeteer/blob/main/third_party/README.md) for details.
Expand Down Expand Up @@ -251,7 +251,7 @@ BINARY=<path-to-executable> npm run test:chrome:headless # Or npm run test:firef
```

If a test is expected to fail on certain configurations or became flaky, update [TestExpectations.json](https://github.com/puppeteer/puppeteer/blob/main/test/TestExpectations.json)
to reflect that. See more info about TestExpectations.json in [`utils/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/utils/mochaRunner).
to reflect that. See more info about TestExpectations.json in [`tools/mochaRunner`](https://github.com/puppeteer/puppeteer/tree/main/tools/mochaRunner).

## API Coverage

Expand All @@ -269,7 +269,7 @@ The following steps are needed to update the Chromium version.

1. Find a suitable Chromium revision
Not all revisions have builds for all platforms, so we need to find one that does.
To do so, run `utils/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `utils/check_availability.js -help` for more options).
To do so, run `tools/check_availability.js -rd` to find the latest suitable `dev` Chromium revision (see `tools/check_availability.js -help` for more options).
1. Update `src/revisions.ts` with the found revision number.
1. Update `versions.js` with the new Chromium-to-Puppeteer version mapping and update `lastMaintainedChromiumVersion` with the latest stable Chrome version.
1. Run `npm run check:protocol-revision`.
Expand All @@ -281,11 +281,11 @@ The following steps are needed to update the Chromium version.

### Bisecting upstream changes

Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `utils/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:
Sometimes, performing a Chromium roll causes tests to fail. To figure out the cause, you need to bisect Chromium revisions to figure out the earliest possible revision that changed the behavior. The script in `tools/bisect.js` can be helpful here. Given a pattern for one or more unit tests, it will automatically bisect the current range:

```sh
node utils/bisect.js --good 686378 --bad 706915 script.js
node utils/bisect.js --unit-test Response.fromCache
node tools/bisect.js --good 686378 --bad 706915 script.js
node tools/bisect.js --unit-test Response.fromCache
```

By default, it will use the Chromium revision in `src/revisions.ts` from the `main` branch and from the working tree to determine the range to bisect.
Expand Down