web-test-runner doesn't report suite failures or top suite failures #31946
Description
Command
test
Is this a regression?
- Yes, this behavior used to work in the previous version
The previous version in which this bug was not present was
No response
Description
The Angular web-test-runner integration appears to ignore errors that occur in afterAll functions or at moments when no spec is executing. This is basically the same bug as blueprintui/web-test-runner-jasmine#25.
Minimal Reproduction
I found this via code inspection. I haven't been able to get Angular's web-test-runner integration working at all. The only setup instructions I've found are from the Angular 17.1 announcement. They're incomplete, bit-rotted, and look like they might not have worked even when new.
That said, unless there's a second reporter hiding out somewhere in Angular's integration that handles suite-level failures, I'm pretty confident that adding this spec file to a working Angular web-test-runner setup will reproduce the problem:
describe('a suite', function() {
it('a spec', function() {});
afterAll(function() {
throw new Error('nope');
});
});
afterAll(function() {
throw new Error('nope');
});
The output should indicate that both the top suite and the "a suite" suite failed with the corresponding exceptions. Based on reading the code I think that Angular's implementation, like blueprintui/web-test-runner-jasmine, will indicate an overall status of failed without providing any specifics about the failures.
Exception or Error
Your Environment
/ \ _ __ __ _ _ _| | __ _ _ __ / ___| | |_ _|
/ △ \ | '_ \ / _` | | | | |/ _` | '__| | | | | | |
/ ___ \| | | | (_| | |_| | | (_| | | | |___| |___ | |
/_/ \_\_| |_|\__, |\__,_|_|\__,_|_| \____|_____|___|
|___/
Angular CLI: 20.3.12
Node: 20.19.0
Package Manager: npm 10.8.2
OS: darwin arm64
Angular: 20.3.13
... common, compiler, compiler-cli, core, forms
... platform-browser, router
Package Version
---------------------------------------
@angular-devkit/architect 0.2003.12
@angular-devkit/build-angular 20.3.12
@angular-devkit/core 20.3.12
@angular-devkit/schematics 20.3.12
@angular/build 20.3.12
@angular/cli 20.3.12
@schematics/angular 20.3.12
rxjs 7.8.2
typescript 5.9.3
Anything else relevant?
Jasmine reports these failures via the failedExpectations property of the suiteDone and jasmineDone reporter events. Angular's integration ignores failedExpectations in jasmineDone and doesn't implement suiteDone at all.
These error paths have been present since jasmine-core 2.1 and jasmine-core 2.5 respectively. Failing to report them is one of the most common bugs I see in third party reporters.