New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestBed.configureTestingModule Performance Issue #12409

Open
ollwenjones opened this Issue Oct 20, 2016 · 89 comments

Comments

@ollwenjones

ollwenjones commented Oct 20, 2016

I'm submitting a ... (check one with "x")

[ ] bug report 
[x] feature request
[ ] support request 

Current behavior

Importing modules into TestBed.configureTestingModule can slow tests down substantially. We have a 'shared module' with parts, that lots of our components use, including a third party library or two. It is very convenient to import this module as a rather than cherry pick in the tests, however testing a small component yesterday, I saw test bootstrap taking two whole seconds and had to cherry-pick dependencies to make tests run reliably.

Expected behavior

TestBed.configureTestingModule should be performant.

Minimal reproduction of the problem with instructions

In this plunker

  1. go to the src/simple.spec.ts and comment and un-comment the SharedModule import into configureTestingModule.
  2. Observe time it takes for test to run.

I am seeing a jump from 0.079s to 0.241s. (~= 3x slower). Multiply that by 5 cases and this simple test takes a whole second. Make the shared module larger and you have real problems.

What is the motivation / use case for changing the behavior?

  1. Slow tests hurt TDD workflow
  2. Slow enough tests disconnect karma from the browser and fail the suite.

Please tell us about your environment:

Win10, VsCode, Webpack,

  • Angular version: 2.0.X

    Yes.

  • Browser: Locally, PhantomJS. Plunk tested in Chrome.

  • Language: TypeScript (though tests are transpiled to ES5 prior to running, naturally)

Note: Even as I write this, I wonder if it's a reasonable expectation for large modules to go through TestBed quickly? Maybe I'm just doing something wrong architecturally, (use multiple, smaller shared modules, not one big one, etc.) However, tests are time-consuming enough without having to track down every individual dependency of a more complex component just to get started.

@vicb vicb added the comp: testing label Oct 21, 2016

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Oct 28, 2016

Seems like the goal is really to have a fresh component instance each time, and for the most part the testing module is going to be static for a particular .spec file. It also seems like configuring the module at each iteration is the part that is really adding on the time. A better pattern could be configuring the module once, then testModule.createComponent multiple times.

TestBed.configureTestingModule once and calling TestBed.createComponent on that module multiple times doesn't work, though... maybe I'm missing an obvious way to preserve module state between iterations?

Edit: FWIW I tried logging the timestamp and it looks like most of the time passes during TestBed.createComponent, not TestBed.configureTestingModule - however I still suspect it's related to module configuration, since adding things to imported modules adds to the time, whether or not those imports are used in the component being tested.

ollwenjones commented Oct 28, 2016

Seems like the goal is really to have a fresh component instance each time, and for the most part the testing module is going to be static for a particular .spec file. It also seems like configuring the module at each iteration is the part that is really adding on the time. A better pattern could be configuring the module once, then testModule.createComponent multiple times.

TestBed.configureTestingModule once and calling TestBed.createComponent on that module multiple times doesn't work, though... maybe I'm missing an obvious way to preserve module state between iterations?

Edit: FWIW I tried logging the timestamp and it looks like most of the time passes during TestBed.createComponent, not TestBed.configureTestingModule - however I still suspect it's related to module configuration, since adding things to imported modules adds to the time, whether or not those imports are used in the component being tested.

@vmandy

This comment has been minimized.

Show comment
Hide comment
@vmandy

vmandy Nov 15, 2016

I have a noticed unit test execution performance issues that seem directly related to this issue.

I have an ionic 2 / angular 2 app. When trying to unit test my components using TestBed and ComponentFixture classes, there is easily 1-2 seconds spent in the setup of each test that seems directly related to the imports: [IonicModule.forRoot(MyComponent)] that I've added to my testing module via the TestBed.configureTestingModule({...}) method.

When I remove the import, my tests run much faster, but they obviously fail because my component's template no longer compiles.

vmandy commented Nov 15, 2016

I have a noticed unit test execution performance issues that seem directly related to this issue.

I have an ionic 2 / angular 2 app. When trying to unit test my components using TestBed and ComponentFixture classes, there is easily 1-2 seconds spent in the setup of each test that seems directly related to the imports: [IonicModule.forRoot(MyComponent)] that I've added to my testing module via the TestBed.configureTestingModule({...}) method.

When I remove the import, my tests run much faster, but they obviously fail because my component's template no longer compiles.

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Nov 15, 2016

@vmandy I've been working around this by doing one or more of:

  1. spending more time curating just what I need in the test module
  2. mocking the component dependencies that are time-consuming to bootstrap
  3. overriding the template of the component I'm testing so it doesn't depend on much.

All of which are time consuming and sub-optimal 👎 (except from the point of view that I am really forced to isolate the unit I am testing?)

It seems like there are some theoretical optimizations to this at least, because I know these components are not taking 1-2 secs to bootstrap at run-time.

ollwenjones commented Nov 15, 2016

@vmandy I've been working around this by doing one or more of:

  1. spending more time curating just what I need in the test module
  2. mocking the component dependencies that are time-consuming to bootstrap
  3. overriding the template of the component I'm testing so it doesn't depend on much.

All of which are time consuming and sub-optimal 👎 (except from the point of view that I am really forced to isolate the unit I am testing?)

It seems like there are some theoretical optimizations to this at least, because I know these components are not taking 1-2 secs to bootstrap at run-time.

@juliemr juliemr self-assigned this Nov 22, 2016

@FlemmingBehrend

This comment has been minimized.

Show comment
Hide comment
@FlemmingBehrend

FlemmingBehrend Dec 1, 2016

Iam having similar problems. Thought it was caused by wallaby, but it seems to be the TestBed.

wallabyjs/public#885

FlemmingBehrend commented Dec 1, 2016

Iam having similar problems. Thought it was caused by wallaby, but it seems to be the TestBed.

wallabyjs/public#885

@mlakmal

This comment has been minimized.

Show comment
Hide comment
@mlakmal

mlakmal Dec 15, 2016

I am seeing the same issue. Any unit test with component compilation is taking close to 1sec...

mlakmal commented Dec 15, 2016

I am seeing the same issue. Any unit test with component compilation is taking close to 1sec...

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Dec 15, 2016

Unfortunately I was pulled off the Angular 2 project and haven't looked at this in quite some time. It seems like a lot of the complications of TestBed (test-module setup, generally, not just performance) could be avoided by an approach similar to the enzyme.shallow render approach in the React ecosystem. - where just one unit is rendered and none of it's dependencies - the same thing can be accomplished by over-riding an Angular component's template in the .spec, but that is often tedious/time-consuming as well.

ollwenjones commented Dec 15, 2016

Unfortunately I was pulled off the Angular 2 project and haven't looked at this in quite some time. It seems like a lot of the complications of TestBed (test-module setup, generally, not just performance) could be avoided by an approach similar to the enzyme.shallow render approach in the React ecosystem. - where just one unit is rendered and none of it's dependencies - the same thing can be accomplished by over-riding an Angular component's template in the .spec, but that is often tedious/time-consuming as well.

@mlakmal

This comment has been minimized.

Show comment
Hide comment
@mlakmal

mlakmal Dec 15, 2016

we have close to 2000 test cases getting executed just under 1 min in Angular 1 where Angular 2 takes 1min to execute just 100 test cases. It would be nice if we can get a resolution for this issue as it hinders the testing capabilities in large applications.

mlakmal commented Dec 15, 2016

we have close to 2000 test cases getting executed just under 1 min in Angular 1 where Angular 2 takes 1min to execute just 100 test cases. It would be nice if we can get a resolution for this issue as it hinders the testing capabilities in large applications.

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Dec 15, 2016

@mlakmal I think there are lots of folks relying heavily on e2e tests, or just testing functional logic in component classes

let component = new ComponentClass(mockDep1, mockDep2);
expect(component.doAddition(2,2)).toBe(4);

Which doesn't test any of the template rendering (template logic)... but it does allow easy testing of class logic at least

ollwenjones commented Dec 15, 2016

@mlakmal I think there are lots of folks relying heavily on e2e tests, or just testing functional logic in component classes

let component = new ComponentClass(mockDep1, mockDep2);
expect(component.doAddition(2,2)).toBe(4);

Which doesn't test any of the template rendering (template logic)... but it does allow easy testing of class logic at least

@mlakmal

This comment has been minimized.

Show comment
Hide comment
@mlakmal

mlakmal Dec 15, 2016

@ollwenjones thanks, i am going to try that out. i really doesn't want to test any template logic since most of that gets covered by our automated tests, so just testing the component class code should be enough.

mlakmal commented Dec 15, 2016

@ollwenjones thanks, i am going to try that out. i really doesn't want to test any template logic since most of that gets covered by our automated tests, so just testing the component class code should be enough.

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Dec 15, 2016

@mlakmal sweet! Makes me happy to contribute something that actually helps someone. 🍻 - FWIW I'm often torn about template-rendering unit tests, because it's so easy to slip into testing the framework.

ollwenjones commented Dec 15, 2016

@mlakmal sweet! Makes me happy to contribute something that actually helps someone. 🍻 - FWIW I'm often torn about template-rendering unit tests, because it's so easy to slip into testing the framework.

@michaelbromley

This comment has been minimized.

Show comment
Hide comment
@michaelbromley

michaelbromley Dec 15, 2016

I'm in the process of upgrading a pretty big project from rc.4 to 2.3, so it's the first time we're using the NgModule approach.

We have ~1000 unit tests, with maybe 1/2 of those testing components. Prior to upgrade, tests run pretty quickly - around 30s.

After upgrade, they take at leat twice as long, but the worst thing is that they seem kind of "flaky" - Karma will periodically lost the connection to Chrome and often I will need to refresh Chrome multiple times. I suspect that something is timing out but haven't isolated it yet.

@ollwenjones regarding "shallow" component tests, did you know about the NO_ERRORS_SCHEMA that you can set in the testing module?

import { NO_ERRORS_SCHEMA} from '@angular/core';
// ...
TestBed.configureTestingModule({
  declarations: [ /*... whatever */ ],
  schemas: [NO_ERRORS_SCHEMA]
});

This means that the compiler just ignores any elements it does not recognize, meaning you do not need to declare all the components used in the template of the component under test.

michaelbromley commented Dec 15, 2016

I'm in the process of upgrading a pretty big project from rc.4 to 2.3, so it's the first time we're using the NgModule approach.

We have ~1000 unit tests, with maybe 1/2 of those testing components. Prior to upgrade, tests run pretty quickly - around 30s.

After upgrade, they take at leat twice as long, but the worst thing is that they seem kind of "flaky" - Karma will periodically lost the connection to Chrome and often I will need to refresh Chrome multiple times. I suspect that something is timing out but haven't isolated it yet.

@ollwenjones regarding "shallow" component tests, did you know about the NO_ERRORS_SCHEMA that you can set in the testing module?

import { NO_ERRORS_SCHEMA} from '@angular/core';
// ...
TestBed.configureTestingModule({
  declarations: [ /*... whatever */ ],
  schemas: [NO_ERRORS_SCHEMA]
});

This means that the compiler just ignores any elements it does not recognize, meaning you do not need to declare all the components used in the template of the component under test.

@marclaval

This comment has been minimized.

Show comment
Hide comment
@marclaval

marclaval Dec 15, 2016

Member

Found the same issue in Angular's tests: #13500
I was able to improve the situation by creating bespoke testing modules for each test.

Member

marclaval commented Dec 15, 2016

Found the same issue in Angular's tests: #13500
I was able to improve the situation by creating bespoke testing modules for each test.

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Dec 15, 2016

@michaelbromley 😮 I had no idea about NO_ERRORS_SCHEMA would have / will probably save me hours - also an opportunity to hopefully speed up some of these tests, as the leaner the test-module the faster TestBed seems to go.

ollwenjones commented Dec 15, 2016

@michaelbromley 😮 I had no idea about NO_ERRORS_SCHEMA would have / will probably save me hours - also an opportunity to hopefully speed up some of these tests, as the leaner the test-module the faster TestBed seems to go.

@FlemmingBehrend

This comment has been minimized.

Show comment
Hide comment
@FlemmingBehrend

FlemmingBehrend Dec 15, 2016

@michaelbromley that also helped me out a lot. Removing MaterialModule.forRoot() from my imports really speed up my tests. 👍

FlemmingBehrend commented Dec 15, 2016

@michaelbromley that also helped me out a lot. Removing MaterialModule.forRoot() from my imports really speed up my tests. 👍

@awerlang

This comment has been minimized.

Show comment
Hide comment
@awerlang

awerlang Feb 26, 2017

Contributor

Ideally, we should be able to call TestBed.configureTestingModule() inside a beforeAll(). It just doesn't work because of this line. This is reseting the modules before each test. Perhaps it shouldn't matter, but it does because Angular spends quite a lot of time compiling components in JiT mode. Reseting it throws away this information. Recording a profiling session on karma window led me to this conclusion.

IMO, this should be done:

  1. Be able to configure a module in a beforeAll();
  2. Cache compiled components

Another way to fix that is making reset testing module explicit, but this may break user code.

Contributor

awerlang commented Feb 26, 2017

Ideally, we should be able to call TestBed.configureTestingModule() inside a beforeAll(). It just doesn't work because of this line. This is reseting the modules before each test. Perhaps it shouldn't matter, but it does because Angular spends quite a lot of time compiling components in JiT mode. Reseting it throws away this information. Recording a profiling session on karma window led me to this conclusion.

IMO, this should be done:

  1. Be able to configure a module in a beforeAll();
  2. Cache compiled components

Another way to fix that is making reset testing module explicit, but this may break user code.

@gonzofish

This comment has been minimized.

Show comment
Hide comment
@gonzofish

gonzofish Mar 8, 2017

Is it clear that this isn't something to do with karma-phantom? The tests fly in Chrome for me, but Phantom takes at least a second per test. I've mocked the dependencies out as much as possible and they still run slow in Phantom.

gonzofish commented Mar 8, 2017

Is it clear that this isn't something to do with karma-phantom? The tests fly in Chrome for me, but Phantom takes at least a second per test. I've mocked the dependencies out as much as possible and they still run slow in Phantom.

@Necroskillz

This comment has been minimized.

Show comment
Hide comment
@Necroskillz

Necroskillz Mar 8, 2017

phantom is very slow for me (using karma-webpack, but this is a different issue. I use Chrome now, and you can visually see as the tests run when there are tests that use a lot of components that have to be compiled it gets stuck for a second and runs slowly until it hits service tests which run fast.

Configuring module in beforeAll is not so convenient, because i want to create new mocks after each run and have them (and other providers) in a fresh state.

Caching the compiled components is the way to go imo.

Necroskillz commented Mar 8, 2017

phantom is very slow for me (using karma-webpack, but this is a different issue. I use Chrome now, and you can visually see as the tests run when there are tests that use a lot of components that have to be compiled it gets stuck for a second and runs slowly until it hits service tests which run fast.

Configuring module in beforeAll is not so convenient, because i want to create new mocks after each run and have them (and other providers) in a fresh state.

Caching the compiled components is the way to go imo.

@philipooo

This comment has been minimized.

Show comment
Hide comment
@philipooo

philipooo Mar 9, 2017

philipooo commented Mar 9, 2017

@michalmo

This comment has been minimized.

Show comment
Hide comment
@michalmo

michalmo Mar 13, 2017

AOT support would help here too.

michalmo commented Mar 13, 2017

AOT support would help here too.

@thymikee

This comment has been minimized.

Show comment
Hide comment
@thymikee

thymikee Apr 4, 2017

@philipooo with slight modifications to some of Jasmine references (jasmine.createSpyObj() specifically), you can also run your tests without any browser – by using Jest instead of Karma.

For as few as 35 files with total 100 tests Jest has been able to speedup my tests 2.5x (vs Karma on Chrome). Tests are now running in parallel, completely isolated, not to mention superior watch mode with instant feedback.

I've written about it here, if anybody's curious: https://www.xfive.co/blog/testing-angular-faster-jest/.

There's also an issue to integrate it directly into CLI (e.g. under a flag: ng test --jest): angular/angular-cli#4543.

thymikee commented Apr 4, 2017

@philipooo with slight modifications to some of Jasmine references (jasmine.createSpyObj() specifically), you can also run your tests without any browser – by using Jest instead of Karma.

For as few as 35 files with total 100 tests Jest has been able to speedup my tests 2.5x (vs Karma on Chrome). Tests are now running in parallel, completely isolated, not to mention superior watch mode with instant feedback.

I've written about it here, if anybody's curious: https://www.xfive.co/blog/testing-angular-faster-jest/.

There's also an issue to integrate it directly into CLI (e.g. under a flag: ng test --jest): angular/angular-cli#4543.

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Apr 4, 2017

@thymikee I'm super excited to hear that. I've had muuuch better experience testing React code with Jest than I ever have with Angular, and I've been persuaded that end-users of trustworthy frameworks don't need to do as much browser-testing.

Last time I checked Jest was using jsdom which did not play nice with zone.js is this no longer the case? 🤞 I'll have to read your blog.

ollwenjones commented Apr 4, 2017

@thymikee I'm super excited to hear that. I've had muuuch better experience testing React code with Jest than I ever have with Angular, and I've been persuaded that end-users of trustworthy frameworks don't need to do as much browser-testing.

Last time I checked Jest was using jsdom which did not play nice with zone.js is this no longer the case? 🤞 I'll have to read your blog.

@thymikee

This comment has been minimized.

Show comment
Hide comment
@thymikee

thymikee Apr 4, 2017

So jsdom is not the problem. It's just that tests must run in Zone.js context, which is done by this little lib: jest-zone-patch

thymikee commented Apr 4, 2017

So jsdom is not the problem. It's just that tests must run in Zone.js context, which is done by this little lib: jest-zone-patch

@jsgoupil

This comment has been minimized.

Show comment
Hide comment
@jsgoupil

jsgoupil Apr 10, 2017

Testing with HTML components is just a pain... I have about 500 unit tests that are 90% looking at the DOM, it takes me 40 minutes to run them all... at this rate, Chrome crashes when it reaches a RAM level too high... the result is that I have to batch my tests by 50. Painful as hell...

There are definitely some memory leaks here and there...

jsgoupil commented Apr 10, 2017

Testing with HTML components is just a pain... I have about 500 unit tests that are 90% looking at the DOM, it takes me 40 minutes to run them all... at this rate, Chrome crashes when it reaches a RAM level too high... the result is that I have to batch my tests by 50. Painful as hell...

There are definitely some memory leaks here and there...

@yairgo

This comment has been minimized.

Show comment
Hide comment
@yairgo

yairgo May 3, 2017

bueller?

yairgo commented May 3, 2017

bueller?

@oferh

This comment has been minimized.

Show comment
Hide comment
@oferh

oferh Jun 11, 2017

Found out that using FormsModule considerably slows down compileComponents mainly using ngModel.

For example here is a plunker which has a component with 10 inputs and it takes about 3 times more when using ngModel.

oferh commented Jun 11, 2017

Found out that using FormsModule considerably slows down compileComponents mainly using ngModel.

For example here is a plunker which has a component with 10 inputs and it takes about 3 times more when using ngModel.

@dblVs

This comment has been minimized.

Show comment
Hide comment
@dblVs

dblVs Jun 12, 2017

@juliemr Hey, I wanted to do a feature request to add a schema that does shallow rendering. Shall keep the stuff here (since what @ollwenjones wrote pretty much is what i want) or create a new issue?
Basically it's:

  • Being able to instantiate the child components, BUT not it's template. (this is a bit more complex than it sound)
  • Still having errors when bound to wrong property on the component (unlike error schema)

At the moment I'm creating stub components in my project to create blank components and have them implement same interface to make them consistent the inputs and outputs and makes it easier to my users to test the template of their component. It becomes really difficult once you include stuff like ContentChildren logic combined with templateOutlet directive usage or transclusion.

Possible solution is use normal schema ot 1st component then for the child components use the NO_ERROR_SCHEMA.

dblVs commented Jun 12, 2017

@juliemr Hey, I wanted to do a feature request to add a schema that does shallow rendering. Shall keep the stuff here (since what @ollwenjones wrote pretty much is what i want) or create a new issue?
Basically it's:

  • Being able to instantiate the child components, BUT not it's template. (this is a bit more complex than it sound)
  • Still having errors when bound to wrong property on the component (unlike error schema)

At the moment I'm creating stub components in my project to create blank components and have them implement same interface to make them consistent the inputs and outputs and makes it easier to my users to test the template of their component. It becomes really difficult once you include stuff like ContentChildren logic combined with templateOutlet directive usage or transclusion.

Possible solution is use normal schema ot 1st component then for the child components use the NO_ERROR_SCHEMA.

@anjmao

This comment has been minimized.

Show comment
Hide comment
@anjmao

anjmao Mar 21, 2018

I think the problem is that initTestEnvironment is called on every tests run so if you are on watch mode the only good way would be to cache compiled templates. Angular currently is not exposing cache https://github.com/angular/angular/blob/master/packages/compiler/src/jit/compiler.ts#L41

If it would be possible to give custom cache store implementation for testing then we could store cache on global karma object something like this.

__karma__.compiledTemplateCache = __karma__.compiledTemplateCache || new Map<Type, CompiledTemplate>();

@Injectable()
class TemplateCacheStore {
  getTemplateCache(meta) {
    // ... get from global __karma__.compiledTemplateCache
  }

  setTemplateCache(meta) {
    // set to global __karma__.compiledTemplateCache
  }
}

getTestBed().initTestEnvironment(
  BrowserDynamicTestingModule.configure({ templatesCacheStore: TemplateCacheStore }),
  platformBrowserDynamicTesting()
);

anjmao commented Mar 21, 2018

I think the problem is that initTestEnvironment is called on every tests run so if you are on watch mode the only good way would be to cache compiled templates. Angular currently is not exposing cache https://github.com/angular/angular/blob/master/packages/compiler/src/jit/compiler.ts#L41

If it would be possible to give custom cache store implementation for testing then we could store cache on global karma object something like this.

__karma__.compiledTemplateCache = __karma__.compiledTemplateCache || new Map<Type, CompiledTemplate>();

@Injectable()
class TemplateCacheStore {
  getTemplateCache(meta) {
    // ... get from global __karma__.compiledTemplateCache
  }

  setTemplateCache(meta) {
    // set to global __karma__.compiledTemplateCache
  }
}

getTestBed().initTestEnvironment(
  BrowserDynamicTestingModule.configure({ templatesCacheStore: TemplateCacheStore }),
  platformBrowserDynamicTesting()
);
@brian428

This comment has been minimized.

Show comment
Hide comment
@brian428

brian428 Mar 21, 2018

The hack by @vvasabi sounds like it would work, at least in some situations. But going through and changing hundreds of spec files would be very time consuming. And there's no telling how changes to future versions of Angular will break the workaround.

The bottom line is that needing to apply hacks like that is really just absurd. It simply shouldn't take 10 minutes to run UI unit tests. It basically cancels out the whole point of TDD.

What's even more frustrating is that all of these problems would be nullified if we could just tell Angular to re-use the AoT summaries and/or re-use the compiled test module. This issue has been ongoing for over 18 months, and it's baffling to me that the Angular folks just seem to be ignoring it.

I don't understand all the inner workings of the TestBed and the test compiler, but from what I can tell, this should be pretty easy to do for someone who actually understands the internals. Angular seems to already create AoT summaries in memory, so exposing a flag to cache and reuse them within a spec file instead of re-creating them each time seems like a straightforward change.

@vikerman was any further thought put into documentation or examples for using a template cache or re-using the AoT summaries?

brian428 commented Mar 21, 2018

The hack by @vvasabi sounds like it would work, at least in some situations. But going through and changing hundreds of spec files would be very time consuming. And there's no telling how changes to future versions of Angular will break the workaround.

The bottom line is that needing to apply hacks like that is really just absurd. It simply shouldn't take 10 minutes to run UI unit tests. It basically cancels out the whole point of TDD.

What's even more frustrating is that all of these problems would be nullified if we could just tell Angular to re-use the AoT summaries and/or re-use the compiled test module. This issue has been ongoing for over 18 months, and it's baffling to me that the Angular folks just seem to be ignoring it.

I don't understand all the inner workings of the TestBed and the test compiler, but from what I can tell, this should be pretty easy to do for someone who actually understands the internals. Angular seems to already create AoT summaries in memory, so exposing a flag to cache and reuse them within a spec file instead of re-creating them each time seems like a straightforward change.

@vikerman was any further thought put into documentation or examples for using a template cache or re-using the AoT summaries?

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Mar 21, 2018

I agree... I feel validated by the fact that this issue isn't just affecting us because of some mistake of mine, and it's encouraging to see the community put forth work-arounds, but also disappointing that we're left with implementing hacks with no road-map from core team members... I basically gave up trying to get my team to write unit tests for components... hopefully integration tests will serve us well.

ollwenjones commented Mar 21, 2018

I agree... I feel validated by the fact that this issue isn't just affecting us because of some mistake of mine, and it's encouraging to see the community put forth work-arounds, but also disappointing that we're left with implementing hacks with no road-map from core team members... I basically gave up trying to get my team to write unit tests for components... hopefully integration tests will serve us well.

@michaelbromley

This comment has been minimized.

Show comment
Hide comment
@michaelbromley

michaelbromley Apr 23, 2018

I've made a feature request at #22797 which addresses this issue too.

It would:

  • provide a safer "shallow render" experience than NO_ERRORS_SCHEMA.
  • prevent the problem of having to declare (or mock) the entire dependency graph of the component under test.
  • keep guarantees of correctness in respect to the selector and inputs/outputs of child components, making the unit tests resistant to breakage via refactorings.

Please take a look and provide feedback, since I think this would vastly improve the testing experience of Angular apps.

michaelbromley commented Apr 23, 2018

I've made a feature request at #22797 which addresses this issue too.

It would:

  • provide a safer "shallow render" experience than NO_ERRORS_SCHEMA.
  • prevent the problem of having to declare (or mock) the entire dependency graph of the component under test.
  • keep guarantees of correctness in respect to the selector and inputs/outputs of child components, making the unit tests resistant to breakage via refactorings.

Please take a look and provide feedback, since I think this would vastly improve the testing experience of Angular apps.

@brian428

This comment has been minimized.

Show comment
Hide comment
@brian428

brian428 Apr 23, 2018

@michaelbromley This is a great idea and I love it, but it seems like a parallel option rather than a real fix for the underlying problem (that the components are recompiled over and over again when they don't need to be). Regardless, I'm on board with the mocking idea 100%.

brian428 commented Apr 23, 2018

@michaelbromley This is a great idea and I love it, but it seems like a parallel option rather than a real fix for the underlying problem (that the components are recompiled over and over again when they don't need to be). Regardless, I'm on board with the mocking idea 100%.

@brian428

This comment has been minimized.

Show comment
Hide comment
@brian428

brian428 Apr 23, 2018

Also just to show how solvable this problem seems to be, I set up a global function that is called in the beforeAll() block of all of my tests. It does a stupid, ridiculous hack to replace the logic in resetTestingModule().

I wouldn't recommend that anyone actually use this, but it seems to work fine and it speeds things up by nearly 40%. My point isn't to offer a solution for anyone to use, but simply to show how easy it seems to be to vastly speed things up just by reusing the compiled components:

let tb: any = getTestBed();

// Switch back to original resetTestingModule() function if it exists
if( ( window as any )[ "__testCache" ] ) tb.resetTestingModule = ( window as any )[ "__testCache" ].realResetTestingModule;

// Do a reset on the testing module.
TestBed.resetTestingModule();

// Store reference to original resetTestingModule() function.
let realResetTestingModule = tb.resetTestingModule;
( window as any )[ "__testCache" ] = { realResetTestingModule: tb.resetTestingModule };

// Replace original resetTestingModule() with a custom version that re-uses the moduleFactory and compiler.
// This cuts the test execution time by roughly 40%.
tb.resetTestingModule = () => {
  let mf = tb._moduleFactory;
  let compiler = tb._compiler;
  realResetTestingModule.apply( tb );
  tb._moduleFactory = mf;
  tb._compiler = compiler;
};

brian428 commented Apr 23, 2018

Also just to show how solvable this problem seems to be, I set up a global function that is called in the beforeAll() block of all of my tests. It does a stupid, ridiculous hack to replace the logic in resetTestingModule().

I wouldn't recommend that anyone actually use this, but it seems to work fine and it speeds things up by nearly 40%. My point isn't to offer a solution for anyone to use, but simply to show how easy it seems to be to vastly speed things up just by reusing the compiled components:

let tb: any = getTestBed();

// Switch back to original resetTestingModule() function if it exists
if( ( window as any )[ "__testCache" ] ) tb.resetTestingModule = ( window as any )[ "__testCache" ].realResetTestingModule;

// Do a reset on the testing module.
TestBed.resetTestingModule();

// Store reference to original resetTestingModule() function.
let realResetTestingModule = tb.resetTestingModule;
( window as any )[ "__testCache" ] = { realResetTestingModule: tb.resetTestingModule };

// Replace original resetTestingModule() with a custom version that re-uses the moduleFactory and compiler.
// This cuts the test execution time by roughly 40%.
tb.resetTestingModule = () => {
  let mf = tb._moduleFactory;
  let compiler = tb._compiler;
  realResetTestingModule.apply( tb );
  tb._moduleFactory = mf;
  tb._compiler = compiler;
};
@ChrisGibb

This comment has been minimized.

Show comment
Hide comment
@ChrisGibb

ChrisGibb Apr 25, 2018

@vmandy (I know I'm late) .. but facing this problem of slow unit, out of memory errors as well

The two things I did to get around this are:

  1. All variables that are used in beforeEach, are declared at the top of the file inside the first describe. e.g.
describe("FooComponent", () => {
 let var0: TypeFoo;
 let var1: TypeFoo;
 let var2: TypeFoo;

 beforeEach(() => {
   var0 = // something big
   var1 = // something big
   var2 = // something big
 })
 afterEach(() => {
   var0 = undefined
   var1 = undefined
   var2 = undefined
 })

 // my other 'describe', 'beforeEach' and 'it' blocks afterwards
});

This will fixed my out of memory problems.

  1. To speed things up. Replace all your IonicModule bits with NO_SCHEMA_ERRORS e.g:

Replace:

imports: [
 IonicModule.forRoot(component),
]

with:

schemas: [
 NO_ERRORS_SCHEMA
]

And you're all good!

ChrisGibb commented Apr 25, 2018

@vmandy (I know I'm late) .. but facing this problem of slow unit, out of memory errors as well

The two things I did to get around this are:

  1. All variables that are used in beforeEach, are declared at the top of the file inside the first describe. e.g.
describe("FooComponent", () => {
 let var0: TypeFoo;
 let var1: TypeFoo;
 let var2: TypeFoo;

 beforeEach(() => {
   var0 = // something big
   var1 = // something big
   var2 = // something big
 })
 afterEach(() => {
   var0 = undefined
   var1 = undefined
   var2 = undefined
 })

 // my other 'describe', 'beforeEach' and 'it' blocks afterwards
});

This will fixed my out of memory problems.

  1. To speed things up. Replace all your IonicModule bits with NO_SCHEMA_ERRORS e.g:

Replace:

imports: [
 IonicModule.forRoot(component),
]

with:

schemas: [
 NO_ERRORS_SCHEMA
]

And you're all good!

@kedar9444

This comment has been minimized.

Show comment
Hide comment
@kedar9444

kedar9444 May 22, 2018

@vvasabi @BurningDog @brian428 : is this hack still works with angular 5.1.3?
because i have configured the test suit same way and able to execute the single spec file. but when i try to execute all the spec files, the first one get passed and all other test cases get failed with following error.

Failed: Uncaught (in promise): Error: Illegal state: Could not load the summary for directive AccountDetailsComponent.

if not what are possible workarounds available for me, because our current test suit taking around 10 to 12 min to execute the test cases and if we multiple this number with head count of resources we are spending to much time just to execute the test cases.

kedar9444 commented May 22, 2018

@vvasabi @BurningDog @brian428 : is this hack still works with angular 5.1.3?
because i have configured the test suit same way and able to execute the single spec file. but when i try to execute all the spec files, the first one get passed and all other test cases get failed with following error.

Failed: Uncaught (in promise): Error: Illegal state: Could not load the summary for directive AccountDetailsComponent.

if not what are possible workarounds available for me, because our current test suit taking around 10 to 12 min to execute the test cases and if we multiple this number with head count of resources we are spending to much time just to execute the test cases.

@vvasabi

This comment has been minimized.

Show comment
Hide comment
@vvasabi

vvasabi May 22, 2018

@kedar9444 My project is on Angular 5.2.0, and I have not had to change the way I implemented my hack since I originally published it last year. If you could share some of your code that reproduces the issue, maybe people can take a look.

vvasabi commented May 22, 2018

@kedar9444 My project is on Angular 5.2.0, and I have not had to change the way I implemented my hack since I originally published it last year. If you could share some of your code that reproduces the issue, maybe people can take a look.

@kedar9444

This comment has been minimized.

Show comment
Hide comment
@kedar9444

kedar9444 May 22, 2018

@vvasabi : thanks for responding i can breath now!!! lets see what i have done in my code.

I have created one configuration file for all spec files :


export const configureTestSuite = () => {
    const testBedApi: any = getTestBed();
    const originReset = TestBed.resetTestingModule;

    TestBed.resetTestingModule();
    TestBed.resetTestingModule = () => TestBed;

    afterEach(() => {
        testBedApi._activeFixtures.forEach((fixture: ComponentFixture<any>) => fixture.destroy());
        testBedApi._instantiated = false;
    });

    afterAll(() => {
        TestBed.resetTestingModule = originReset;
        TestBed.resetTestingModule();
    });
}

then used the same in actual test case :



describe('Component: xxxComponent', () => {
    configureTestSuite();

    beforeAll(done => (async () => {

        TestBed.configureTestingModule({
            imports: [SharedModule, BrowserModule...],
            declarations: [xxxComponent, yyyComponent],
            schemas: [CUSTOM_ELEMENTS_SCHEMA],
            providers: [
                { provide: xxxService, useClass: xxxServiceStub }
                ...
            ],

        })

        await TestBed.compileComponents();
        
    })().then(done).catch(done.fail));

    describe('Component: templateFormComponent', () => {

        let xxxfixture: ComponentFixture<xxxComponent>;
        let xComponent: xxxComponent;

        let yyyfixture: ComponentFixture<yyyComponent>;
        let yComponent: yyyComponent;

        let xService: xxxService
        let el: HTMLElement;

        beforeEach(() => {

            xxxfixture = TestBed.createComponent(xxxComponent);
            xComponent = xxxfixture.componentInstance;

            xService = TestBed.get(xxxService);
        });
    });
});


with this implementation if i tried to run any single spec file [with fdescribe] it gives proper result, but if i tried to run all the spec files in one go the configuration runs the first spec file correctly but through error for all other spec files.

and the error is :

Failed: Uncaught (in promise): Error: Illegal state: Could not load the summary for directive xxxComponent.

if you need more data please mention in comment. thanks in advance. :)

here are some more details:


Angular CLI: 1.7.4
Node: 9.11.1
OS: win32 x64
Angular: 5.2.4
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.7.4
@angular-devkit/core: 0.0.29
@schematics/package-update: 0.3.2
typescript: 2.4.2
webpack: error

kedar9444 commented May 22, 2018

@vvasabi : thanks for responding i can breath now!!! lets see what i have done in my code.

I have created one configuration file for all spec files :


export const configureTestSuite = () => {
    const testBedApi: any = getTestBed();
    const originReset = TestBed.resetTestingModule;

    TestBed.resetTestingModule();
    TestBed.resetTestingModule = () => TestBed;

    afterEach(() => {
        testBedApi._activeFixtures.forEach((fixture: ComponentFixture<any>) => fixture.destroy());
        testBedApi._instantiated = false;
    });

    afterAll(() => {
        TestBed.resetTestingModule = originReset;
        TestBed.resetTestingModule();
    });
}

then used the same in actual test case :



describe('Component: xxxComponent', () => {
    configureTestSuite();

    beforeAll(done => (async () => {

        TestBed.configureTestingModule({
            imports: [SharedModule, BrowserModule...],
            declarations: [xxxComponent, yyyComponent],
            schemas: [CUSTOM_ELEMENTS_SCHEMA],
            providers: [
                { provide: xxxService, useClass: xxxServiceStub }
                ...
            ],

        })

        await TestBed.compileComponents();
        
    })().then(done).catch(done.fail));

    describe('Component: templateFormComponent', () => {

        let xxxfixture: ComponentFixture<xxxComponent>;
        let xComponent: xxxComponent;

        let yyyfixture: ComponentFixture<yyyComponent>;
        let yComponent: yyyComponent;

        let xService: xxxService
        let el: HTMLElement;

        beforeEach(() => {

            xxxfixture = TestBed.createComponent(xxxComponent);
            xComponent = xxxfixture.componentInstance;

            xService = TestBed.get(xxxService);
        });
    });
});


with this implementation if i tried to run any single spec file [with fdescribe] it gives proper result, but if i tried to run all the spec files in one go the configuration runs the first spec file correctly but through error for all other spec files.

and the error is :

Failed: Uncaught (in promise): Error: Illegal state: Could not load the summary for directive xxxComponent.

if you need more data please mention in comment. thanks in advance. :)

here are some more details:


Angular CLI: 1.7.4
Node: 9.11.1
OS: win32 x64
Angular: 5.2.4
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.7.4
@angular-devkit/core: 0.0.29
@schematics/package-update: 0.3.2
typescript: 2.4.2
webpack: error

@kedar9444

This comment has been minimized.

Show comment
Hide comment
@kedar9444

kedar9444 commented May 23, 2018

here is the code to reproduce the issue.

https://github.com/kedar9444/unit-test-performance

@vvasabi

This comment has been minimized.

Show comment
Hide comment
@vvasabi

vvasabi May 23, 2018

@kedar9444 The Github repo was helpful. All I did to fix your issue was to have this in configure-test-suit.ts:

beforeAll(() => {
  TestBed.resetTestingModule();
  TestBed.resetTestingModule = () => TestBed;
});

You need those 2 lines in a beforeAll block. Hope this helps.

vvasabi commented May 23, 2018

@kedar9444 The Github repo was helpful. All I did to fix your issue was to have this in configure-test-suit.ts:

beforeAll(() => {
  TestBed.resetTestingModule();
  TestBed.resetTestingModule = () => TestBed;
});

You need those 2 lines in a beforeAll block. Hope this helps.

@blacksonic

This comment has been minimized.

Show comment
Hide comment
@blacksonic

blacksonic May 29, 2018

After adding the beforeAll part I get this error: Cannot configure the test module when the test module has already been instantiated.
I call configureTestingModule in my beforeEach hooks.

blacksonic commented May 29, 2018

After adding the beforeAll part I get this error: Cannot configure the test module when the test module has already been instantiated.
I call configureTestingModule in my beforeEach hooks.

@vvasabi

This comment has been minimized.

Show comment
Hide comment
@vvasabi

vvasabi May 30, 2018

@blacksonic The idea behind this hack is that time is saved by not having to call configureTestingModule once per it block to recompile components. As such, the complaint that the test module has already been instantiated will go away if you only call configureTestingModule once per the outermost describe block.

vvasabi commented May 30, 2018

@blacksonic The idea behind this hack is that time is saved by not having to call configureTestingModule once per it block to recompile components. As such, the complaint that the test module has already been instantiated will go away if you only call configureTestingModule once per the outermost describe block.

@oocx

This comment has been minimized.

Show comment
Hide comment
@oocx

oocx Jul 20, 2018

Contributor

I created a pull request for this problem last year but never got any feedback from the Angular team. (#17710). We are using code very similar to that pull request at my employer in an Angular app with more than 1.000 unit tests.

Contributor

oocx commented Jul 20, 2018

I created a pull request for this problem last year but never got any feedback from the Angular team. (#17710). We are using code very similar to that pull request at my employer in an Angular app with more than 1.000 unit tests.

@giggio

This comment has been minimized.

Show comment
Hide comment
@giggio

giggio Jul 24, 2018

Testing in Angular is really problematic right now because of this performance issue. We are also on a 1000+ tests app, and because of this hack we were able to go from about 30 minutes to about 3 minutes for a full test run. This is really bad, a 30m test run is unacceptable. And even 3m is bad for unit tests for just 1000+ tests.

The problem lies on all the rendering and the dependency on a browser. I hope angular switches to something like JSDom ASAP (and away from Karma and Jasmine), otherwise tests will never be as fast as they have to be.

giggio commented Jul 24, 2018

Testing in Angular is really problematic right now because of this performance issue. We are also on a 1000+ tests app, and because of this hack we were able to go from about 30 minutes to about 3 minutes for a full test run. This is really bad, a 30m test run is unacceptable. And even 3m is bad for unit tests for just 1000+ tests.

The problem lies on all the rendering and the dependency on a browser. I hope angular switches to something like JSDom ASAP (and away from Karma and Jasmine), otherwise tests will never be as fast as they have to be.

@vvasabi

This comment has been minimized.

Show comment
Hide comment
@vvasabi

vvasabi Jul 26, 2018

@giggio You can already use JSDom with Angular unit tests via Jest. Look for thymikee’s comments in this thread.

vvasabi commented Jul 26, 2018

@giggio You can already use JSDom with Angular unit tests via Jest. Look for thymikee’s comments in this thread.

@awerlang

This comment has been minimized.

Show comment
Hide comment
@awerlang

awerlang Jul 27, 2018

Contributor

The problem isn't in the real vs fake DOM approach. jsdom-based tests with Jest can cut running time to half. Still too slow for even a dozen tests. I'm afraid Jest has still a long way to go to take advantage of parallelism & fake DOM which in theory looks much better than current implementations. The community would benefit from this of course.

To achieve a tenfold decrease in running time Angular needs to allow the testing module to be configured in a beforeAll() block. Right now it's just naively discarding it after each test. This should be prioritized until caching compilation of sorts takes place. We don't need the best solution now, but we deserve a non-naïve solution at least until something better is created.

So do we want to have a 2x or 10x speedup? And how long are we willing to wait?

Contributor

awerlang commented Jul 27, 2018

The problem isn't in the real vs fake DOM approach. jsdom-based tests with Jest can cut running time to half. Still too slow for even a dozen tests. I'm afraid Jest has still a long way to go to take advantage of parallelism & fake DOM which in theory looks much better than current implementations. The community would benefit from this of course.

To achieve a tenfold decrease in running time Angular needs to allow the testing module to be configured in a beforeAll() block. Right now it's just naively discarding it after each test. This should be prioritized until caching compilation of sorts takes place. We don't need the best solution now, but we deserve a non-naïve solution at least until something better is created.

So do we want to have a 2x or 10x speedup? And how long are we willing to wait?

@mattem

This comment has been minimized.

Show comment
Hide comment
@mattem

mattem Jul 27, 2018

Something that has worked well for us when testing our component library is splitting it up into smaller libraries where possible, then running these library tests in parallel on CI nodes, while also using Karma plugins such as karma-parallel to parallelize even further.

That, coupled with overriding of the resetTestingModule and managing the reset manually as previously mentioned in this thread, has helped reduce times across 6000+ specs enough while we wait for a Bazel / caching build based solution.

mattem commented Jul 27, 2018

Something that has worked well for us when testing our component library is splitting it up into smaller libraries where possible, then running these library tests in parallel on CI nodes, while also using Karma plugins such as karma-parallel to parallelize even further.

That, coupled with overriding of the resetTestingModule and managing the reset manually as previously mentioned in this thread, has helped reduce times across 6000+ specs enough while we wait for a Bazel / caching build based solution.

@blackholegalaxy

This comment has been minimized.

Show comment
Hide comment
@blackholegalaxy

blackholegalaxy Jul 27, 2018

we should'nt have to rely on CI tricks to speedup the tests. I think @awerlang is right: Angular should act in a way the tests are ran properly without rebilding the whole component if the setup can be consistent between the tests.

blackholegalaxy commented Jul 27, 2018

we should'nt have to rely on CI tricks to speedup the tests. I think @awerlang is right: Angular should act in a way the tests are ran properly without rebilding the whole component if the setup can be consistent between the tests.

@brian428

This comment has been minimized.

Show comment
Hide comment
@brian428

brian428 Jul 27, 2018

The frustrating thing about this is that, as I showed earlier in this thread, the logic to simply hold onto the compiled test module is trivial. I just have no idea how to "properly" change the actual Angular code within TestBed or Compiler to allow this, or I'd have sent a pull request long ago.

I realize that there are probably edge cases where my solution won't work (e.g. where individual specs are changing/overriding different parts of the test module), but having a flag to turn the compilation result caching on or off would also be a simple addition.

I'm really just totally baffled that the problem has been this bad for this long, yet no one on the team seems to be prioritizing this.

brian428 commented Jul 27, 2018

The frustrating thing about this is that, as I showed earlier in this thread, the logic to simply hold onto the compiled test module is trivial. I just have no idea how to "properly" change the actual Angular code within TestBed or Compiler to allow this, or I'd have sent a pull request long ago.

I realize that there are probably edge cases where my solution won't work (e.g. where individual specs are changing/overriding different parts of the test module), but having a flag to turn the compilation result caching on or off would also be a simple addition.

I'm really just totally baffled that the problem has been this bad for this long, yet no one on the team seems to be prioritizing this.

@abeymg

This comment has been minimized.

Show comment
Hide comment
@abeymg

abeymg Sep 26, 2018

Any update on this? Is any effort being made to speed up test execution when using the TestBed ? How do larger projects at Google that use Angular deal with the slow tests ?

abeymg commented Sep 26, 2018

Any update on this? Is any effort being made to speed up test execution when using the TestBed ? How do larger projects at Google that use Angular deal with the slow tests ?

@kyliau

This comment has been minimized.

Show comment
Hide comment
@kyliau

kyliau Sep 26, 2018

Member

I've just LGTM @oocx #17710.
In Google, tests are done with AOT by providing the NgSummaries to TestModuleMetadata
This is the recommended way to get the best performance, but we understand that it might not be easy to get hold of the NgSummary files.
With Ivy coming out soon AOT will be the default. In the meantime, please try out the suggestions in the PR.

Member

kyliau commented Sep 26, 2018

I've just LGTM @oocx #17710.
In Google, tests are done with AOT by providing the NgSummaries to TestModuleMetadata
This is the recommended way to get the best performance, but we understand that it might not be easy to get hold of the NgSummary files.
With Ivy coming out soon AOT will be the default. In the meantime, please try out the suggestions in the PR.

@mattem

This comment has been minimized.

Show comment
Hide comment
@mattem

mattem Sep 26, 2018

@kyliau Do you have any examples of passing AOT summaries here?

mattem commented Sep 26, 2018

@kyliau Do you have any examples of passing AOT summaries here?

@abeymg

This comment has been minimized.

Show comment
Hide comment
@abeymg

abeymg Sep 27, 2018

@kyliau Thanks for the response. I am already using the one of the workarounds mentioned earlier in this thread. My question was more about when the official support for this would land in the core. I'll follow the PR to see when it makes it out.

Question about what you mentioned about Ivy. With Ivy making AOT the default, does the workflow for tests using TestBed change?

abeymg commented Sep 27, 2018

@kyliau Thanks for the response. I am already using the one of the workarounds mentioned earlier in this thread. My question was more about when the official support for this would land in the core. I'll follow the PR to see when it makes it out.

Question about what you mentioned about Ivy. With Ivy making AOT the default, does the workflow for tests using TestBed change?

@alex-kel

This comment has been minimized.

Show comment
Hide comment
@alex-kel

alex-kel Sep 28, 2018

Hello everyone! I am trying to implement this approach with configureTestSuite function.
Did someone face issue when with this approach Karma says that I am running 'Empty test suite'?

Here the test code - https://gist.github.com/alex-kel/e549a845e37b00678a17da7340432683

@awerlang @vvasabi probably you guys can help me?

alex-kel commented Sep 28, 2018

Hello everyone! I am trying to implement this approach with configureTestSuite function.
Did someone face issue when with this approach Karma says that I am running 'Empty test suite'?

Here the test code - https://gist.github.com/alex-kel/e549a845e37b00678a17da7340432683

@awerlang @vvasabi probably you guys can help me?

@kekel87

This comment has been minimized.

Show comment
Hide comment
@kekel87

kekel87 Sep 29, 2018

Hello,
In my team, we just used ng-bullet to accelerate the tests of our different projects.
This allowed impressing gains (ex: 2m30s to 30s), without any problem for the moment.

kekel87 commented Sep 29, 2018

Hello,
In my team, we just used ng-bullet to accelerate the tests of our different projects.
This allowed impressing gains (ex: 2m30s to 30s), without any problem for the moment.

@alex-kel

This comment has been minimized.

Show comment
Hide comment
@alex-kel

alex-kel Sep 29, 2018

alex-kel commented Sep 29, 2018

@ollwenjones

This comment has been minimized.

Show comment
Hide comment
@ollwenjones

ollwenjones Oct 8, 2018

@kekel87 that's brilliant that someone took on that problem with a tidy library API! When I opened the issue, I never dreamed it would come 2 years later from a 3rd party, though.

ollwenjones commented Oct 8, 2018

@kekel87 that's brilliant that someone took on that problem with a tidy library API! When I opened the issue, I never dreamed it would come 2 years later from a 3rd party, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment