Compilation errors don't cause non-zero exit codes in ng test #2778

JohannesRudolph · 2016-10-19T12:15:22Z

Please provide us with the following information:

OS?

Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)
OS X El Capitan

Versions.

Please run ng --version. If there's nothing outputted, please run in a Terminal: node --version and paste the result here:
angular-cli: 1.0.0-beta.17
node: 6.3.0
os: darwin x64
Repro steps.

Was this an app that wasn't created using the CLI? What change did you do on your code? etc.
Create a angular cli project, create a compilation error in a spec file that doesn't fail at runtime (e.g. omit constructor parameter) and run ng test

The log given by the failure.

Normally this include a stack trace and some more information.

For example, this output of a test run of ng test:

ERROR in [default] /Users/jr/dev/console/console/src/app/context/context.service.spec.ts:17:12 
Supplied parameters do not match any signature of call target.

ERROR in [default] /Users/jr/dev/console/console/src/app/context/location.spec.ts:8:11 
Supplied parameters do not match any signature of call target.
19 10 2016 14:05:24.377:INFO [karma]: Karma v1.2.0 server started at http://localhost:9876/
19 10 2016 14:05:24.380:INFO [launcher]: Launching browser Chrome with unlimited concurrency
19 10 2016 14:05:24.511:INFO [launcher]: Starting browser Chrome
19 10 2016 14:05:25.882:INFO [Chrome 53.0.2785 (Mac OS X 10.11.6)]: Connected on socket /#pLo6aNiQB7OJVlxvAAAA with id 31332214
Chrome 53.0.2785 (Mac OS X 10.11.6): Executed 10 of 10 SUCCESS (0.77 secs / 0.754 secs)
iDevBook01:console jr$ $?
-bash: 0: command not found

Mention any other details that might be useful.

It would be useful if angular-cli captured the error code when compiling typescript code and failed early. ng build doesn't build tests, so it's not useful either.

Thanks! We'll be in touch soon.

The text was updated successfully, but these errors were encountered:

kylecordes · 2016-10-21T19:12:27Z

This also affects, for example, ng build. An argument could be made that ng serve should carry on even with typescript errors, but build and test presumably should fail if the code is broken.

filipesilva · 2016-10-27T15:26:24Z

Commands like ng test, ng serve and ng build --watch that run in watch mode don't usually exit with an error. This is by design.

Non-watch commands like ng test --watch=false, ng build and ng e2e do exit with an error code though, so I think what you're looking for is ng test --watch=false.

For instance, I removed AppComponent parameters in let fixture = TestBed.createComponent(AppComponent);, which causes the Supplied parameters do not match any signature of call target. error:

kamik@T460p MINGW64 /D/sandbox/master-proj (master)
$ ng test --watch=false


ERROR in [default] D:\sandbox\master-proj\src\app\app.component.spec.ts:16:18
Supplied parameters do not match any signature of call target.
27 10 2016 16:22:03.288:INFO [karma]: Karma v1.2.0 server started at http://localhost:9876/
27 10 2016 16:22:03.290:INFO [launcher]: Launching browser Chrome with unlimited concurrency
27 10 2016 16:22:03.297:INFO [launcher]: Starting browser Chrome
27 10 2016 16:22:04.628:INFO [Chrome 54.0.2840 (Windows 10 0.0.0)]: Connected on socket /#cdGZJk8pod052mpYAAAA with id 64527908
Chrome 54.0.2840 (Windows 10 0.0.0) App: MasterProj should create the app FAILED
        Failed: Cannot create the component undefined as it was not imported into the testing module!
        Error: Cannot create the component undefined as it was not imported into the testing module!
            at TestBed.createComponent (webpack:///D:/sandbox/master-proj/~/@angular/core/bundles/core-testing.umd.js:846:0 <- src/test.ts:3863:23)
            at Function.TestBed.createComponent (webpack:///D:/sandbox/master-proj/~/@angular/core/bundles/core-testing.umd.js:673:0 <- src/test.ts:3690:33)
            at webpack:///D:/sandbox/master-proj/src/app/app.component.spec.ts:16:26 <- src/test.ts:21280:41
            at ZoneDelegate.invoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/zone.js:232:0 <- src/test.ts:20064:26)
            at AsyncTestZoneSpec.onInvoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/async-test.js:49:0 <- src/test.ts:12928:39)
            at ProxyZoneSpec.onInvoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/proxy.js:76:0 <- src/test.ts:13620:39)
            at ZoneDelegate.invoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/zone.js:231:0 <- src/test.ts:20063:32)
            at Zone.runGuarded (webpack:///D:/sandbox/master-proj/~/zone.js/dist/zone.js:128:0 <- src/test.ts:19960:47)
            at runInTestZone (webpack:///D:/sandbox/master-proj/~/@angular/core/bundles/core-testing.umd.js:105:0 <- src/test.ts:3122:29)
            at Object.<anonymous> (webpack:///D:/sandbox/master-proj/~/@angular/core/bundles/core-testing.umd.js:48:0 <- src/test.ts:3065:17)
Chrome 54.0.2840 (Windows 10 0.0.0): Executed 1 of 3 (1 FAILED) (0 secs / 0.093 secs)
Chrome 54.0.2840 (Windows 10 0.0.0) App: MasterProj should create the app FAILED
        Failed: Cannot create the component undefined as it was not imported into the testing module!
        Error: Cannot create the component undefined as it was not imported into the testing module!
            at TestBed.createComponent (webpack:///D:/sandbox/master-proj/~/@angular/core/bundles/core-testing.umd.js:846:0 <- src/test.ts:3863:23)
            at Function.TestBed.createComponent (webpack:///D:/sandbox/master-proj/~/@angular/core/bundles/core-testing.umd.js:673:0 <- src/test.ts:3690:33)
            at webpack:///D:/sandbox/master-proj/src/app/app.component.spec.ts:16:26 <- src/test.ts:21280:41
            at ZoneDelegate.invoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/zone.js:232:0 <- src/test.ts:20064:26)
            at AsyncTestZoneSpec.onInvoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/async-test.js:49:0 <- src/test.ts:12928:39)
            at ProxyZoneSpec.onInvoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/proxy.js:76:0 <- src/test.ts:13620:39)
            at ZoneDelegate.invoke (webpack:///D:/sandbox/master-proj/~/zone.js/dist/zone.js:231:0 <- src/test.ts:20063:32)
            at Zone.runGuarded (webpack:///D:/sandbox/master-proj/~/zone.js/dist/zone.js:128:0 <- src/test.ts:19960:47)
            at runInTestZone (webpack:///D:/sandbox/master-proj/~/@angular/core/bundles/core-testing.umd.js:105:0 <- src/test.ts:3122:29)
Chrome 54.0.2840 (Windows 10 0.0.0): Executed 3 of 3 (1 FAILED) (0.211 secs / 0.18 secs)

kamik@T460p MINGW64 /D/sandbox/master-proj (master)
$ echo $?
1

@kylecordes I don't understand what you're talking about. Have you tried ng build with a compilation error and did not get an error exit code? If so, please open a new issue with a reproduction.

JohannesRudolph · 2016-10-28T13:41:20Z

Thanks for responding and your great work on angular-cli @filipesilva

sorry for not being more specific in the original request. The error is a build error not a test error. I did indeed execute the test with ng test --watch=false and observed that the exit code is zero (see below for a full reproduction).

I agree about returning nothing meaningful in --watch=true, but the original issue still stands. Is that enough for reopening this issue?

iDevBook01:console jr$ ng test --watch=false
Could not start watchman; falling back to NodeWatcher for file system events.
Visit http://ember-cli.com/user-guide/#watchman for more info.

WARNING in ./src/app/core/location.ts
[5, 85]: missing whitespace

 @ ./src/app/core/index.ts 9:451-472
 @ ./src/app/app.component.ts
 @ ./src/app/app.component.spec.ts
 @ ./src \.spec\.ts
 @ ./src/test.ts

ERROR in [default] /Users/jr/dev/console/console/src/app/core/context.service.spec.ts:24:12 
Supplied parameters do not match any signature of call target.

28 10 2016 15:36:40.593:INFO [karma]: Karma v1.2.0 server started at http://localhost:9876/
28 10 2016 15:36:40.595:INFO [launcher]: Launching browser Chrome with unlimited concurrency
28 10 2016 15:36:40.601:INFO [launcher]: Starting browser Chrome
28 10 2016 15:36:42.874:INFO [Chrome 54.0.2840 (Mac OS X 10.11.6)]: Connected on socket /#qitPU5ll3ETn1TolAAAA with id 88987585
Chrome 54.0.2840 (Mac OS X 10.11.6): Executed 3 of 18 SUCCESS (0 secs / 0.516 secs)
....
iDevBook01:console jr$ $?
-bash: 0: command not found

filipesilva · 2016-11-03T00:30:57Z

A build error should definitely make ng test --watch=false exit with an error code. In my test it did, but in your case it did not even though they should have been very similar.

Can you give me the exact code I can use to reproduce?

fabian-dev · 2016-11-18T13:11:38Z

I can confirm this.

In a new angular-cli Project create a spec that has TS errors, but is valid JS. Example app.component.spec.ts:

describe('App: Ngnew', () => {
  beforeEach(() => {
    TestBed.configureTestingModule({
      declarations: [
        AppComponent
      ],
    });
  });

  export interface UseInSut {
    someValue: number;
  }

  it('should create the app', async(() => {
    let useInSut: UseInSut = {
      nonExisitung: 1;
    }
    expect(true).toBeTruthy();
  }));

});

Although not valid TS, ng test will pass with RC=0.

$ ng test --watch=false

ERROR in [default] C:\DEV\TRY\ngnew\src\app\app.component.spec.ts:23:6
Type '{ nonExisitung: number; }' is not assignable to type 'UseInSut'.
  Object literal may only specify known properties, and 'nonExisitung' does not exist in type 'UseInSut'.

ERROR in [default] C:\DEV\TRY\ngnew\src\app\app.component.spec.ts:23:21
',' expected.
18 11 2016 14:10:09.677:INFO [karma]: Karma v1.2.0 server started at http://localhost:9876/
18 11 2016 14:10:09.679:INFO [launcher]: Launching browser Chrome with unlimited concurrency
18 11 2016 14:10:09.687:INFO [launcher]: Starting browser Chrome
18 11 2016 14:10:12.066:INFO [Chrome 54.0.2840 (Windows 7 0.0.0)]: Connected on socket /#YuC0qEmPAM5CAwZoAAAA with id 63478678
Chrome 54.0.2840 (Windows 7 0.0.0): Executed 1 of 1 SUCCESS (0.009 secs / 0.005 secs)

afterwards $ echo $? gives me 0.

I would expect ng test to fail on TS errors.

JohannesRudolph · 2017-01-23T09:08:28Z

Any news on this?

Fix angular#2778 Fix angular#3424

Fix #2778 Fix #3424

smkart · 2019-04-24T05:02:02Z

@hansl What version of cli come with this fix?

In my case I am using Jenkins to "ng test" , Because of exit code is always 0 Jenkins pipeline never goes to failed

We are using cli v1.7.3 and we don't have any roadmap to upgrade cli version for now, I am not sure in which version this issue got fixed

Thanks in advance

Regards,
Mani
Applied Materials

umutau · 2019-04-24T15:23:53Z

in my case ng test --watch=false --browsers=ChromeHeadless && ng build is working as expected

angular-automatic-lock-bot · 2019-09-09T12:41:02Z

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

filipesilva closed this as completed Oct 27, 2016

filipesilva reopened this Nov 3, 2016

filipesilva added the needs: repro steps We cannot reproduce the issue with the information given label Nov 3, 2016

JohannesRudolph mentioned this issue Dec 8, 2016

"ng test" does not bail on type definition errors #3424

Closed

filipesilva self-assigned this Dec 12, 2016

filipesilva added command: test needs: investigation Requires some digging to determine if action is needed type: bug/fix and removed needs: repro steps We cannot reproduce the issue with the information given labels Dec 12, 2016

JohannesRudolph mentioned this issue Jan 9, 2017

ng test does not honor tsconfig.json compilation options #3911

Closed

filipesilva added a commit to filipesilva/angular-cli that referenced this issue Mar 19, 2017

fix(@angular/cli): fix error handling on test

f1e27db

Fix angular#2778 Fix angular#3424

filipesilva mentioned this issue Mar 19, 2017

fix(@angular/cli): fix test typings #5504

Merged

filipesilva added a commit to filipesilva/angular-cli that referenced this issue Mar 19, 2017

fix(@angular/cli): fix error handling on test

1e72de0

Fix angular#2778 Fix angular#3424

filipesilva added a commit to filipesilva/angular-cli that referenced this issue Mar 19, 2017

fix(@angular/cli): fix error handling on test

055a4b7

Fix angular#2778 Fix angular#3424

filipesilva added a commit to filipesilva/angular-cli that referenced this issue Mar 20, 2017

fix(@angular/cli): fix error handling on test

58299dd

Fix angular#2778 Fix angular#3424

filipesilva added a commit to filipesilva/angular-cli that referenced this issue Mar 20, 2017

fix(@angular/cli): fix error handling on test

8ab3954

Fix angular#2778 Fix angular#3424

hansl closed this as completed in #5504 Mar 20, 2017

hansl pushed a commit that referenced this issue Mar 20, 2017

fix(@angular/cli): fix error handling on test

9cda847

Fix #2778 Fix #3424

angular-automatic-lock-bot bot locked and limited conversation to collaborators Sep 9, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compilation errors don't cause non-zero exit codes in ng test #2778

Compilation errors don't cause non-zero exit codes in ng test #2778

JohannesRudolph commented Oct 19, 2016 •

edited

Please provide us with the following information:

Repro steps.

kylecordes commented Oct 21, 2016

filipesilva commented Oct 27, 2016

JohannesRudolph commented Oct 28, 2016

filipesilva commented Nov 3, 2016

fabian-dev commented Nov 18, 2016

JohannesRudolph commented Jan 23, 2017

smkart commented Apr 24, 2019

umutau commented Apr 24, 2019

angular-automatic-lock-bot bot commented Sep 9, 2019

Compilation errors don't cause non-zero exit codes in ng test #2778

Compilation errors don't cause non-zero exit codes in ng test #2778

Comments

JohannesRudolph commented Oct 19, 2016 • edited

Please provide us with the following information:

OS?

Versions.

Repro steps.

The log given by the failure.

Mention any other details that might be useful.

kylecordes commented Oct 21, 2016

filipesilva commented Oct 27, 2016

JohannesRudolph commented Oct 28, 2016

filipesilva commented Nov 3, 2016

fabian-dev commented Nov 18, 2016

JohannesRudolph commented Jan 23, 2017

smkart commented Apr 24, 2019

umutau commented Apr 24, 2019

angular-automatic-lock-bot bot commented Sep 9, 2019

JohannesRudolph commented Oct 19, 2016 •

edited