New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General Testing Help / Advice #191

Open
lathonez opened this Issue Dec 9, 2016 · 40 comments

Comments

Projects
None yet
@lathonez
Owner

lathonez commented Dec 9, 2016

The purpose of this issue is a sticky thread general for help with testing Ionic 2 projects.

E.g. you have our testing framework set up fine but you don't know how to test a particular scenario / component / feature, or you're getting unexpected results from some of your tests.

The correct place for posting such questions is stack overflow. Asking your question on stack will:

  • give you a wider audience
  • give you a better quality answer
  • give you a quicker answer

Please follow these guidelines:

  1. post your question on stack
  2. tag your question with angular2 and angular2-testing. You can tag ionic2 too if your question is about testing some Ionic component
  3. comment on this issue with a link to your question. This will draw our attention to it and we'll answer on stack if we can help. It will also provide a good reference point for others.

Further issues raised on this repo will be referred here.

If you're having trouble setting up testing (e.g. you can't get a single basic test running against your app) please raise an issue here - don't use stack for that

@kamok

This comment has been minimized.

Show comment
Hide comment

kamok commented Jan 2, 2017

@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Jun 8, 2017

Owner
Owner

lathonez commented Jun 8, 2017

@kamok

This comment has been minimized.

Show comment
Hide comment
@kamok

kamok Jun 16, 2017

@lathonez I figured out the new ionic Keyboard from @ionic-native/keyboard. We ended up not using TestUtils when we had Keyboard, and doing the test module configuration inside the spec itself. Is that how you guys did it?

kamok commented Jun 16, 2017

@lathonez I figured out the new ionic Keyboard from @ionic-native/keyboard. We ended up not using TestUtils when we had Keyboard, and doing the test module configuration inside the spec itself. Is that how you guys did it?

@stonelasley

This comment has been minimized.

Show comment
Hide comment
@stonelasley

stonelasley Jul 7, 2017

Contributor

@kamok @lathonez I've been grappling with the problem I think you're describing, when to use TestUtils and when you need a more tailored testmodule. I have a project where I'm using the TestUtil to provide the base "ambient" providers but then I have an additional parameter that allows me to pass in additional providers that I may like a reference to or as additional dependencies. Have you guys come up with a better way?

TestUtils

public static beforeEachCompiler(components: any[], providers: any[]): Promise<{ fixture: any, instance: any }> {
		return TestUtils.configureIonicTestingModule(components, providers)
			.compileComponents().then(() => {
				let fixture: any = TestBed.createComponent(components[0]);
				return {
					fixture: fixture,
					instance: fixture.debugElement.componentInstance,
				};
			});
	}

	public static configureIonicTestingModule(components: Array<any>, componentProviders: any[]): typeof TestBed {
		let coreProviders: any[] = [
			App,
			DomController,
			GestureController,
			{provide: Keyboard, useFactory: () => KeyboardMock.instance()},
			{provide: MenuController, useFactory: () => MenuControllerMock.instance()},
			{provide: Form, useFactory: () => FormMock.instance()},
			{provide: Config, useFactory: () => ConfigMock.instance()},
			{provide: TranslateService, useFactory: (TranslateServiceMock.instance)},
			{provide: Platform, useFactory: () => PlatformMock.instance()},
		];

		let providers: any[] = coreProviders.concat(componentProviders);

		return TestBed.configureTestingModule({
				imports: [IonicModule, CommonModule],
				declarations: [components, TranslatePipeMock ],
				providers: providers
			});
	}

which in use looks something like:

beforeEach(async(() => {

			form = FormGroupMock.instance(true, formVal);
			viewCtrl = ViewControllerMock.instance();
			formBuilder = FormBuilderMock.instance(form);
			settingsSrvc = SettingsServiceMock.instance();
			dateSrvc = DateServiceMock.instance();


			let providers: any[] = [
				{provide: FormBuilder, useFactory: () => formBuilder},
				{provide: SettingsService, useFactory: () => settingsSrvc},
				{provide: ViewController, useFactory: () => viewCtrl},
				{provide: DateService, useFactory: () => dateSrvc}
			];

			return TestUtils.beforeEachCompiler([ReportCreatePage], providers)
				.then(compiled => {

					fixture = compiled.fixture;
					instance = compiled.instance;
					classUnderTest = fixture.componentInstance;

					fixture.detectChanges();
				});
		}
	));
Contributor

stonelasley commented Jul 7, 2017

@kamok @lathonez I've been grappling with the problem I think you're describing, when to use TestUtils and when you need a more tailored testmodule. I have a project where I'm using the TestUtil to provide the base "ambient" providers but then I have an additional parameter that allows me to pass in additional providers that I may like a reference to or as additional dependencies. Have you guys come up with a better way?

TestUtils

public static beforeEachCompiler(components: any[], providers: any[]): Promise<{ fixture: any, instance: any }> {
		return TestUtils.configureIonicTestingModule(components, providers)
			.compileComponents().then(() => {
				let fixture: any = TestBed.createComponent(components[0]);
				return {
					fixture: fixture,
					instance: fixture.debugElement.componentInstance,
				};
			});
	}

	public static configureIonicTestingModule(components: Array<any>, componentProviders: any[]): typeof TestBed {
		let coreProviders: any[] = [
			App,
			DomController,
			GestureController,
			{provide: Keyboard, useFactory: () => KeyboardMock.instance()},
			{provide: MenuController, useFactory: () => MenuControllerMock.instance()},
			{provide: Form, useFactory: () => FormMock.instance()},
			{provide: Config, useFactory: () => ConfigMock.instance()},
			{provide: TranslateService, useFactory: (TranslateServiceMock.instance)},
			{provide: Platform, useFactory: () => PlatformMock.instance()},
		];

		let providers: any[] = coreProviders.concat(componentProviders);

		return TestBed.configureTestingModule({
				imports: [IonicModule, CommonModule],
				declarations: [components, TranslatePipeMock ],
				providers: providers
			});
	}

which in use looks something like:

beforeEach(async(() => {

			form = FormGroupMock.instance(true, formVal);
			viewCtrl = ViewControllerMock.instance();
			formBuilder = FormBuilderMock.instance(form);
			settingsSrvc = SettingsServiceMock.instance();
			dateSrvc = DateServiceMock.instance();


			let providers: any[] = [
				{provide: FormBuilder, useFactory: () => formBuilder},
				{provide: SettingsService, useFactory: () => settingsSrvc},
				{provide: ViewController, useFactory: () => viewCtrl},
				{provide: DateService, useFactory: () => dateSrvc}
			];

			return TestUtils.beforeEachCompiler([ReportCreatePage], providers)
				.then(compiled => {

					fixture = compiled.fixture;
					instance = compiled.instance;
					classUnderTest = fixture.componentInstance;

					fixture.detectChanges();
				});
		}
	));
@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Jul 11, 2017

Owner

The way I do this is defining different "sets" of providers for different usages, and assigning each to a static variable against TestUtils.

Instead of declaring the providers in the beforeEach and passing them in, I pass through this variable TestUtils.ReportProviders which would then add the providers as necessary.

It's basically the same as what you have there, I just have lots of spec and it saves me from importing the additional providers each time.

In practice I think I have four different sets that I use.

Owner

lathonez commented Jul 11, 2017

The way I do this is defining different "sets" of providers for different usages, and assigning each to a static variable against TestUtils.

Instead of declaring the providers in the beforeEach and passing them in, I pass through this variable TestUtils.ReportProviders which would then add the providers as necessary.

It's basically the same as what you have there, I just have lots of spec and it saves me from importing the additional providers each time.

In practice I think I have four different sets that I use.

@euleule

This comment has been minimized.

Show comment
Hide comment
@euleule

euleule Aug 16, 2017

Hi. I have trouble testing a component containg a FAB. Maybe you can give me a hint how to solve this. https://stackoverflow.com/questions/45711222/how-to-test-a-floating-action-button-in-ionic2

euleule commented Aug 16, 2017

Hi. I have trouble testing a component containg a FAB. Maybe you can give me a hint how to solve this. https://stackoverflow.com/questions/45711222/how-to-test-a-floating-action-button-in-ionic2

@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Aug 16, 2017

Owner

The error is coming from UIEventManager: https://github.com/ionic-team/ionic/blob/master/src/gestures/ui-event-manager.ts#L48

this.evts is an array of 1 undefined element, hence the error.

this.evts is set in UIEventManager.listen, which uses Platform: https://github.com/ionic-team/ionic/blob/master/src/gestures/ui-event-manager.ts#L40

So we need to look at our platform mock: https://github.com/stonelasley/ionic-mocks/blob/master/src/angular/platform.ts#L26

Adding this line solves for me. Please raise a PR against ionic-mocks to fix!

instance.registerListener.and.returnValue(() => {});

Thanks

Owner

lathonez commented Aug 16, 2017

The error is coming from UIEventManager: https://github.com/ionic-team/ionic/blob/master/src/gestures/ui-event-manager.ts#L48

this.evts is an array of 1 undefined element, hence the error.

this.evts is set in UIEventManager.listen, which uses Platform: https://github.com/ionic-team/ionic/blob/master/src/gestures/ui-event-manager.ts#L40

So we need to look at our platform mock: https://github.com/stonelasley/ionic-mocks/blob/master/src/angular/platform.ts#L26

Adding this line solves for me. Please raise a PR against ionic-mocks to fix!

instance.registerListener.and.returnValue(() => {});

Thanks

@lathonez lathonez changed the title from General Unit Testing Help / Advice to General Testing Help / Advice Aug 22, 2017

@stonelasley

This comment has been minimized.

Show comment
Hide comment
@stonelasley

stonelasley Oct 8, 2017

Contributor

how is everyone testing classes with asynchronous tasks in their constructor? I've started using the following approach but I'd love to find a better way.

class MyClass { 
 constructor(platform: Platform, service: MyService) {


    this.platform = platform;
    this.translateService = translateService;

    this.initialize();
  }

  private initialize(service: MyService): Promise<any> {
	return service.doSomeAsync()
  }
 }
  
  
 /* Spec */
 
 describe('MyClass', () => {
 
    let serviceMock: any;
	let classUnderTest: MyClass;
	beforeEach(() => {
		classUnderTest = new MyClass(serviceMock);
	);
	
	describe('initialize', done => {
               //initialize can also be protected and then I'll extend the class under test in the spec but this is the lazy approach. 
		classUnderTest['initialize']()
		.then(() => {
			//TEST ASYNC CONSTRUCTOR LOGIC
			done();
		});
	
	});
 }
Contributor

stonelasley commented Oct 8, 2017

how is everyone testing classes with asynchronous tasks in their constructor? I've started using the following approach but I'd love to find a better way.

class MyClass { 
 constructor(platform: Platform, service: MyService) {


    this.platform = platform;
    this.translateService = translateService;

    this.initialize();
  }

  private initialize(service: MyService): Promise<any> {
	return service.doSomeAsync()
  }
 }
  
  
 /* Spec */
 
 describe('MyClass', () => {
 
    let serviceMock: any;
	let classUnderTest: MyClass;
	beforeEach(() => {
		classUnderTest = new MyClass(serviceMock);
	);
	
	describe('initialize', done => {
               //initialize can also be protected and then I'll extend the class under test in the spec but this is the lazy approach. 
		classUnderTest['initialize']()
		.then(() => {
			//TEST ASYNC CONSTRUCTOR LOGIC
			done();
		});
	
	});
 }
@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Oct 8, 2017

Owner

@stonelasley - I do the same as you. Did some research a while back and it seemed the best way. Also keen to improve if possible.

Owner

lathonez commented Oct 8, 2017

@stonelasley - I do the same as you. Did some research a while back and it seemed the best way. Also keen to improve if possible.

@euleule

This comment has been minimized.

Show comment
Hide comment
@euleule

euleule Oct 10, 2017

@stonelasley Another way would be to use the lifecycle events of angular and ionic to decouple object creation and object initialization. In that case you could test the initialization method as you would test any method containing async calls.

euleule commented Oct 10, 2017

@stonelasley Another way would be to use the lifecycle events of angular and ionic to decouple object creation and object initialization. In that case you could test the initialization method as you would test any method containing async calls.

@stonelasley

This comment has been minimized.

Show comment
Hide comment
@stonelasley

stonelasley Oct 10, 2017

Contributor

@euleule that's a good point and that's also an approach I've used. It points out a weakness in my example. I think I should have provided the initialize to the platform.ready callback. Nonetheless, I like your phrase "decouple object creation and initialization".

Contributor

stonelasley commented Oct 10, 2017

@euleule that's a good point and that's also an approach I've used. It points out a weakness in my example. I think I should have provided the initialize to the platform.ready callback. Nonetheless, I like your phrase "decouple object creation and initialization".

@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Oct 12, 2017

Owner

@euleule / @stonelasley - would love a really quick example

Owner

lathonez commented Oct 12, 2017

@euleule / @stonelasley - would love a really quick example

@euleule

This comment has been minimized.

Show comment
Hide comment
@euleule

euleule Oct 13, 2017

@lathonez If async things need to be done during instantiation, I try not to do them when the object is created, but when it is loaded for the first time. This way I have full control over what happens in the tests. Of course you could also supply a specialized class in the TestBed creation, that already mocks the service with default behaviour, so you can use it during object creation.
This shows how I try to decouple creation and initialization.

class MyClass {
	constructor(service: MyService) {
	}
	
	ngOnInit() {
		this.initialize();
	}

	private initialize(): Promise<any> {
		return this.service.doSomeAsync()
	}
}

/* Spec */
describe('MyClass', () => {

	let classUnderTest: MyClass;
	beforeEach(() => {
		let serviceMock: any = Mock.create(MyService);
                serviceMock.doSomeAsync.and.returnValue(Promise.resolve());
		classUnderTest = new MyClass(serviceMock);
	});

	describe('initialize', () => {
		classUnderTest.ngOnInit();
		expect(serviceMock.doSomeAsync).toHaveBeenCalled();
	});
};

This example shows how I create a mock for a service that I want to call in a constructor. I added a second parameter to your TestUtils wich takes a list definitions that go to the provider definition. I use this to keep the dependencies in the test.ts as small as possible and only add global needed components and services. The dependencies for the tests are added in each respective test. This helps to avoid loading transitive dependencies, because of Mocks, that are written wrong and to me the tests appear a little more self-contained.
Edit: I just noticed that is exactly what @stonelasley has described in July.

describe("MyClass", () => {
	beforeEach(async(() => {
		let myServiceMock: MyService = <any>Mock.create(MyService);
		myServiceMock.doSomeAsync.and.returnValue(Promise.resolve());

		TestUtils.beforeEachCompiler([MyClass], [
			{provide: MyService, useValue: pluginSettingsMock}
		]).then((compiled) => {
			fixture = compiled.fixture;
			instance = compiled.instance;
			fixture.detectChanges();
		});
	}));

	... 
});

The Mock object is a little helper I use to create jasmine Spys.

euleule commented Oct 13, 2017

@lathonez If async things need to be done during instantiation, I try not to do them when the object is created, but when it is loaded for the first time. This way I have full control over what happens in the tests. Of course you could also supply a specialized class in the TestBed creation, that already mocks the service with default behaviour, so you can use it during object creation.
This shows how I try to decouple creation and initialization.

class MyClass {
	constructor(service: MyService) {
	}
	
	ngOnInit() {
		this.initialize();
	}

	private initialize(): Promise<any> {
		return this.service.doSomeAsync()
	}
}

/* Spec */
describe('MyClass', () => {

	let classUnderTest: MyClass;
	beforeEach(() => {
		let serviceMock: any = Mock.create(MyService);
                serviceMock.doSomeAsync.and.returnValue(Promise.resolve());
		classUnderTest = new MyClass(serviceMock);
	});

	describe('initialize', () => {
		classUnderTest.ngOnInit();
		expect(serviceMock.doSomeAsync).toHaveBeenCalled();
	});
};

This example shows how I create a mock for a service that I want to call in a constructor. I added a second parameter to your TestUtils wich takes a list definitions that go to the provider definition. I use this to keep the dependencies in the test.ts as small as possible and only add global needed components and services. The dependencies for the tests are added in each respective test. This helps to avoid loading transitive dependencies, because of Mocks, that are written wrong and to me the tests appear a little more self-contained.
Edit: I just noticed that is exactly what @stonelasley has described in July.

describe("MyClass", () => {
	beforeEach(async(() => {
		let myServiceMock: MyService = <any>Mock.create(MyService);
		myServiceMock.doSomeAsync.and.returnValue(Promise.resolve());

		TestUtils.beforeEachCompiler([MyClass], [
			{provide: MyService, useValue: pluginSettingsMock}
		]).then((compiled) => {
			fixture = compiled.fixture;
			instance = compiled.instance;
			fixture.detectChanges();
		});
	}));

	... 
});

The Mock object is a little helper I use to create jasmine Spys.

@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Oct 22, 2017

Owner

@euleule - this is perfect, thanks.

This helps to avoid loading transitive dependencies, because of Mocks, that are written wrong and to me the tests appear a little more self-contained.

I agree this is more correct.

The Mock object is a little helper I use to create jasmine Spys.

^ That is really neat, thanks.

Owner

lathonez commented Oct 22, 2017

@euleule - this is perfect, thanks.

This helps to avoid loading transitive dependencies, because of Mocks, that are written wrong and to me the tests appear a little more self-contained.

I agree this is more correct.

The Mock object is a little helper I use to create jasmine Spys.

^ That is really neat, thanks.

@ibulmer

This comment has been minimized.

Show comment
Hide comment
@ibulmer

ibulmer Oct 27, 2017

@lathonez. I noticed when I run npm test from within a docker container, I get the following error:
**Cannot start Chrome
[1027/150943.732614:ERROR:nacl_helper_linux.cc(311)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
**

If I change the karma.conf.js from
customLaunchers: { ChromeNoSandbox: { base: 'Chrome', flags: ['--no-sandbox'] } }
customLaunchers: { ChromeNoSandbox: { base: 'ChromeHeadless', flags: ['--no-sandbox'] } }

I no longer get the error. Not sure if this is a bug or if it was something wrong in my approach. Thanks for the great repo!

ibulmer commented Oct 27, 2017

@lathonez. I noticed when I run npm test from within a docker container, I get the following error:
**Cannot start Chrome
[1027/150943.732614:ERROR:nacl_helper_linux.cc(311)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
**

If I change the karma.conf.js from
customLaunchers: { ChromeNoSandbox: { base: 'Chrome', flags: ['--no-sandbox'] } }
customLaunchers: { ChromeNoSandbox: { base: 'ChromeHeadless', flags: ['--no-sandbox'] } }

I no longer get the error. Not sure if this is a bug or if it was something wrong in my approach. Thanks for the great repo!

@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Oct 27, 2017

Owner

@ibulmer - thanks for this.

The entire test suite is actually run inside a docker container (by Circle).

The docker image is here: https://hub.docker.com/r/lathonez/clicker/

Can you replicate on that image? If not, what are the differences between that and the image you are using?

Owner

lathonez commented Oct 27, 2017

@ibulmer - thanks for this.

The entire test suite is actually run inside a docker container (by Circle).

The docker image is here: https://hub.docker.com/r/lathonez/clicker/

Can you replicate on that image? If not, what are the differences between that and the image you are using?

@ibulmer

This comment has been minimized.

Show comment
Hide comment
@ibulmer

ibulmer Oct 27, 2017

@lathonez I am running the docker image lathonez/clicker. First I
docker pull lathonez/clicker
then
docker container run lathonez/clicker npm run test

Following this process I get the errors mentioned earlier

If instead I run it with sh, download vim, edit the karma.conf.js, then run npm run test, it works. My commands are below.

docker container run -it lathonez/clicker sh
apt update && apt install vim
vim karma.conf.js
(make the change)
npm run test

I am guessing I need to run it with circle instead of manually?

ibulmer commented Oct 27, 2017

@lathonez I am running the docker image lathonez/clicker. First I
docker pull lathonez/clicker
then
docker container run lathonez/clicker npm run test

Following this process I get the errors mentioned earlier

If instead I run it with sh, download vim, edit the karma.conf.js, then run npm run test, it works. My commands are below.

docker container run -it lathonez/clicker sh
apt update && apt install vim
vim karma.conf.js
(make the change)
npm run test

I am guessing I need to run it with circle instead of manually?

@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez

lathonez Oct 27, 2017

Owner

This is how it's run in circle:

https://github.com/lathonez/clicker/blob/master/.circleci/config.yml#L27-L32

the xvfb run will be the equivalent of what you're doing with headless in karma.conf.js I think

Owner

lathonez commented Oct 27, 2017

This is how it's run in circle:

https://github.com/lathonez/clicker/blob/master/.circleci/config.yml#L27-L32

the xvfb run will be the equivalent of what you're doing with headless in karma.conf.js I think

@ibulmer

This comment has been minimized.

Show comment
Hide comment
@ibulmer

ibulmer Oct 27, 2017

Ah got it thanks!

ibulmer commented Oct 27, 2017

Ah got it thanks!

@Abhishek-Kanitkar

This comment has been minimized.

Show comment
Hide comment
@Abhishek-Kanitkar

Abhishek-Kanitkar Feb 13, 2018

Hi @lathonez,

I have posted my question on Stack.

Here is link.

https://stackoverflow.com/q/48758724/814477

Can anyone please help me ?

Abhishek-Kanitkar commented Feb 13, 2018

Hi @lathonez,

I have posted my question on Stack.

Here is link.

https://stackoverflow.com/q/48758724/814477

Can anyone please help me ?

@lathonez

This comment has been minimized.

Show comment
Hide comment
@lathonez
Owner

lathonez commented Jun 6, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment