-
Notifications
You must be signed in to change notification settings - Fork 12k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(@angular-devkit/build-angular): initial experimental implementat…
…ion of `@web/test-runner` builder
This is a new `@angular-devkit/build-angular:web-test-runner` builder which invokes Web Test Runner to execute unit tests in a real browser.
The implementation calls `application` builder under the hood with some option overrides build the application to a temporary directory and then runs Web Test Runner on the output. This set up is still minimal, but sufficient to run and pass tests in the generated `ng new` application.
The `schema.json` file is directly copied from the `karma` builder, since this is intended to serve as a migration target for users coming from Karma. Most of the options don't actually work yet, which is logged when they are used.
The most interesting part of this change is configuring Jasmine to execute in Web Test Runner. This is done through the `testRunnerHtml` option which allows us to control the HTML page tests are executed on. We use `test_page.html` which very carefully controls the loading process. I opted to make a single `<script type="module">` which dynamic imports all the relevant pieces so the ordering can be directly controlled more easily. This is better than trying to manage multiple `<script>` tags and pass data between them. Ideally everything would be bundled into a single entry point, however this is not feasible due to the way that ordering requirements do not align with typical `import` structure. Jasmine must come before polyfills which must come before the runner which invokes user code. In an ideal world, this ordering relationship would be represented in `import` statements, but this is not practically feasible because Angular CLI doesn't own all the files (`./polyfills.js` is user-defined) and Jasmine's loading must be split into two places so Zone.js can properly patch it.
`jasmine_runner.js` serves the purpose of executing Jasmine tests and reporting their results to Web Test Runner. I tried to write `jasmine_runner.js` in TypeScript and compile it with a `ts_library`. Unfortunately I don't think this is feasible because it needs to import `@web/test-runner-core` at runtime. This dependency has some code generated at runtime in Web Test Runner, meaning we cannot bundle this dependency and must mark it as external and dynamic `import()` the package at runtime. This works fine in native ESM, but compiling with TypeScript outputs CommonJS code by default (and I don't believe our `@build_bazel_rules_nodejs` setup can easily change that), so any `import('@web/test-runner-core')` becomes `require('@web/test-runner-core')` which fails because that package is ESM-only. The `loadEsmModule` trick does work here either because Web Test Runner is applying Node module resolution at serve time, meaning it looks for `import('@web/test-runner-core')` and rewrites it to something like `import('/node_modules/@web/test-runner-core')`. In short, there is no easy syntax which circumvents the TypeScript compiler while also being statically analyzable to Web Test Runner.- Loading branch information
1 parent
7a8bdee
commit 68dae53
Showing
13 changed files
with
1,847 additions
and
64 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
194 changes: 194 additions & 0 deletions
194
packages/angular_devkit/build_angular/src/builders/web-test-runner/index.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,194 @@ | ||
| /** | ||
| * @license | ||
| * Copyright Google LLC All Rights Reserved. | ||
| * | ||
| * Use of this source code is governed by an MIT-style license that can be | ||
| * found in the LICENSE file at https://angular.io/license | ||
| */ | ||
|
|
||
| import { BuilderContext, BuilderOutput, createBuilder } from '@angular-devkit/architect'; | ||
| import type * as WebTestRunner from '@web/test-runner'; | ||
| import { promises as fs } from 'node:fs'; | ||
| import { createRequire } from 'node:module'; | ||
| import path from 'node:path'; | ||
| import { findTestFiles } from '../../utils/test-files'; | ||
| import { buildApplicationInternal } from '../application'; | ||
| import { OutputHashing } from '../browser-esbuild/schema'; | ||
| import { WtrBuilderOptions, normalizeOptions } from './options'; | ||
| import { Schema } from './schema'; | ||
|
|
||
| export default createBuilder( | ||
| async (schema: Schema, ctx: BuilderContext): Promise<BuilderOutput> => { | ||
| ctx.logger.warn( | ||
| 'NOTE: The Web Test Runner builder is currently EXPERIMENTAL and not ready for production use.', | ||
| ); | ||
|
|
||
| // Dynamic import `@web/test-runner` from the user's workspace. As an optional peer dep, it may not be installed | ||
| // and may not be resolvable from `@angular-devkit/build-angular`. | ||
| const require = createRequire(`${ctx.workspaceRoot}/`); | ||
| let wtr: typeof WebTestRunner; | ||
| try { | ||
| wtr = require('@web/test-runner'); | ||
| } catch { | ||
| return { | ||
| success: false, | ||
| // TODO(dgp1130): Display a more accurate message for non-NPM users. | ||
| error: | ||
| 'Web Test Runner is not installed, most likely you need to run `npm install @web/test-runner --save-dev` in your project.', | ||
| }; | ||
| } | ||
|
|
||
| const options = normalizeOptions(schema); | ||
| const testDir = 'dist/test-out'; | ||
|
|
||
| // Parallelize startup work. | ||
| const [testFiles] = await Promise.all([ | ||
| // Glob for files to test. | ||
| findTestFiles(options.include, options.exclude, ctx.workspaceRoot).then((files) => | ||
| Array.from(files).map((file) => path.relative(process.cwd(), file)), | ||
| ), | ||
| // Clean build output path. | ||
| fs.rm(testDir, { recursive: true, force: true }), | ||
| ]); | ||
|
|
||
| // Build the tests and abort on any build failure. | ||
| const buildOutput = await buildTests(testFiles, testDir, options, ctx); | ||
| if (!buildOutput.success) { | ||
| return buildOutput; | ||
| } | ||
|
|
||
| // Run the built tests. | ||
| return await runTests(wtr, `${testDir}/browser`, options); | ||
| }, | ||
| ); | ||
|
|
||
| /** Build all the given test files and write the result to the given output path. */ | ||
| async function buildTests( | ||
| testFiles: string[], | ||
| outputPath: string, | ||
| options: WtrBuilderOptions, | ||
| ctx: BuilderContext, | ||
| ): Promise<BuilderOutput> { | ||
| const entryPoints = new Set([ | ||
| ...testFiles, | ||
| 'jasmine-core/lib/jasmine-core/jasmine.js', | ||
| '@angular-devkit/build-angular/src/builders/web-test-runner/jasmine_runner.js', | ||
| ]); | ||
|
|
||
| // Extract `zone.js/testing` to a separate entry point because it needs to be loaded after Jasmine. | ||
| const [polyfills, hasZoneTesting] = extractZoneTesting(options.polyfills); | ||
| if (hasZoneTesting) { | ||
| entryPoints.add('zone.js/testing'); | ||
| } | ||
|
|
||
| // Build tests with `application` builder, using test files as entry points. | ||
| // Also bundle in Jasmine and the Jasmine runner script, which need to share chunked dependencies. | ||
| const buildOutput = await first( | ||
| buildApplicationInternal( | ||
| { | ||
| entryPoints, | ||
| tsConfig: options.tsConfig, | ||
| outputPath, | ||
| aot: false, | ||
| index: false, | ||
| outputHashing: OutputHashing.None, | ||
| optimization: false, | ||
| externalDependencies: [ | ||
| // Resolved by `@web/test-runner` at runtime with dynamically generated code. | ||
| '@web/test-runner-core', | ||
| ], | ||
| sourceMap: { | ||
| scripts: true, | ||
| styles: true, | ||
| vendor: true, | ||
| }, | ||
| polyfills, | ||
| }, | ||
| ctx, | ||
| ), | ||
| ); | ||
|
|
||
| return buildOutput; | ||
| } | ||
|
|
||
| function extractZoneTesting( | ||
| polyfills: readonly string[], | ||
| ): [polyfills: string[], hasZoneTesting: boolean] { | ||
| const polyfillsWithoutZoneTesting = polyfills.filter( | ||
| (polyfill) => polyfill !== 'zone.js/testing', | ||
| ); | ||
| const hasZoneTesting = polyfills.length !== polyfillsWithoutZoneTesting.length; | ||
|
|
||
| return [polyfillsWithoutZoneTesting, hasZoneTesting]; | ||
| } | ||
|
|
||
| /** Run Web Test Runner on the given directory of bundled JavaScript tests. */ | ||
| async function runTests( | ||
| wtr: typeof WebTestRunner, | ||
| testDir: string, | ||
| options: WtrBuilderOptions, | ||
| ): Promise<BuilderOutput> { | ||
| const testPagePath = path.resolve(__dirname, 'test_page.html'); | ||
| const testPage = await fs.readFile(testPagePath, 'utf8'); | ||
|
|
||
| const runner = await wtr.startTestRunner({ | ||
| config: { | ||
| rootDir: testDir, | ||
| files: [ | ||
| `${testDir}/**/*.js`, | ||
| `!${testDir}/polyfills.js`, | ||
| `!${testDir}/chunk-*.js`, | ||
| `!${testDir}/jasmine.js`, | ||
| `!${testDir}/jasmine_runner.js`, | ||
| `!${testDir}/testing.js`, // `zone.js/testing` | ||
| ], | ||
| testFramework: { | ||
| config: { | ||
| defaultTimeoutInterval: 5_000, | ||
| }, | ||
| }, | ||
| nodeResolve: true, | ||
| port: 9876, | ||
| watch: options.watch ?? false, | ||
|
|
||
| testRunnerHtml: (_testFramework, _config) => testPage, | ||
| }, | ||
| readCliArgs: false, | ||
| readFileConfig: false, | ||
| autoExitProcess: false, | ||
| }); | ||
| if (!runner) { | ||
| throw new Error('Failed to start Web Test Runner.'); | ||
| } | ||
|
|
||
| // Wait for the tests to complete and stop the runner. | ||
| const passed = (await once(runner, 'finished')) as boolean; | ||
| await runner.stop(); | ||
|
|
||
| // No need to return error messages because Web Test Runner already printed them to the console. | ||
| return { success: passed }; | ||
| } | ||
|
|
||
| /** Returns the first item yielded by the given generator and cancels the execution. */ | ||
| async function first<T>(generator: AsyncIterable<T>): Promise<T> { | ||
| for await (const value of generator) { | ||
| return value; | ||
| } | ||
|
|
||
| throw new Error('Expected generator to emit at least once.'); | ||
| } | ||
|
|
||
| /** Listens for a single emission of an event and returns the value emitted. */ | ||
| // eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
| function once<Map extends Record<string, any>, EventKey extends string & keyof Map>( | ||
| emitter: WebTestRunner.EventEmitter<Map>, | ||
| event: EventKey, | ||
| ): Promise<Map[EventKey]> { | ||
| return new Promise((resolve) => { | ||
| const onEmit = (arg: Map[EventKey]): void => { | ||
| emitter.off(event, onEmit); | ||
| resolve(arg); | ||
| }; | ||
| emitter.on(event, onEmit); | ||
| }); | ||
| } |
88 changes: 88 additions & 0 deletions
88
packages/angular_devkit/build_angular/src/builders/web-test-runner/jasmine_runner.js
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| /** | ||
| * @license | ||
| * Copyright Google LLC All Rights Reserved. | ||
| * | ||
| * Use of this source code is governed by an MIT-style license that can be | ||
| * found in the LICENSE file at https://angular.io/license | ||
| */ | ||
|
|
||
| import { getTestBed } from '@angular/core/testing'; | ||
| import { | ||
| BrowserDynamicTestingModule, | ||
| platformBrowserDynamicTesting, | ||
| } from '@angular/platform-browser-dynamic/testing'; | ||
| import { | ||
| getConfig, | ||
| sessionFailed, | ||
| sessionFinished, | ||
| sessionStarted, | ||
| } from '@web/test-runner-core/browser/session.js'; | ||
|
|
||
| /** Executes Angular Jasmine tests in the given environment and reports the results to Web Test Runner. */ | ||
| export async function runJasmineTests(jasmineEnv) { | ||
| const allSpecs = []; | ||
| const failedSpecs = []; | ||
|
|
||
| jasmineEnv.addReporter({ | ||
| specDone(result) { | ||
| const expectations = [...result.passedExpectations, ...result.failedExpectations]; | ||
| allSpecs.push(...expectations.map((e) => ({ name: e.fullName, passed: e.passed }))); | ||
|
|
||
| for (const e of result.failedExpectations) { | ||
| const message = `${result.fullName}\n${e.message}\n${e.stack}`; | ||
| // eslint-disable-next-line no-console | ||
| console.error(message); | ||
| failedSpecs.push({ | ||
| message, | ||
| name: e.fullName, | ||
| stack: e.stack, | ||
| expected: e.expected, | ||
| actual: e.actual, | ||
| }); | ||
| } | ||
| }, | ||
|
|
||
| async jasmineDone(result) { | ||
| // eslint-disable-next-line no-console | ||
| console.log(`Tests ${result.overallStatus}!`); | ||
| await sessionFinished({ | ||
| passed: result.overallStatus === 'passed', | ||
| errors: failedSpecs, | ||
| testResults: { | ||
| name: '', | ||
| suites: [], | ||
| tests: allSpecs, | ||
| }, | ||
| }); | ||
| }, | ||
| }); | ||
|
|
||
| await sessionStarted(); | ||
|
|
||
| // Web Test Runner uses a different HTML page for every test, so we only get one `testFile` for the single `*.js` file we need to execute. | ||
| const { testFile, testFrameworkConfig } = await getConfig(); | ||
| const config = { defaultTimeoutInterval: 60_000, ...(testFrameworkConfig ?? {}) }; | ||
|
|
||
| // eslint-disable-next-line no-undef | ||
| jasmine.DEFAULT_TIMEOUT_INTERVAL = config.defaultTimeoutInterval; | ||
|
|
||
| // Initialize `TestBed` automatically for users. This assumes we already evaluated `zone.js/testing`. | ||
| getTestBed().initTestEnvironment(BrowserDynamicTestingModule, platformBrowserDynamicTesting(), { | ||
| errorOnUnknownElements: true, | ||
| errorOnUnknownProperties: true, | ||
| }); | ||
|
|
||
| // Load the test file and evaluate it. | ||
| try { | ||
| // eslint-disable-next-line no-undef | ||
| await import(new URL(testFile, document.baseURI).href); | ||
|
|
||
| // Execute the test functions. | ||
| // eslint-disable-next-line no-undef | ||
| jasmineEnv.execute(); | ||
| } catch (err) { | ||
| // eslint-disable-next-line no-console | ||
| console.error(err); | ||
| await sessionFailed(err); | ||
| } | ||
| } |
Oops, something went wrong.