Issue running e2e angular tests #17

Closed
jacobkretz opened this Issue Apr 23, 2013 · 17 comments

Comments

Projects
None yet
@jacobkretz

I install the yeoman angular stack today (1.0, angular and karma generator) and while unit tests work out of the box, e2e testing does not. I created a simple e2e test based on the default yeoman angular app (see below) and no matter the configuration I can never get the karma e2e runner to connect to my running site. Any thoughts would be appreciated. Thanks

'use strict';

describe('Controller: MainCtrl', function () {

it('should filter results', function() {
browser().navigateTo('/')
expect(repeater('ul li').count()).toEqual(3);
});

});

@kevin1024

This comment has been minimized.

Show comment
Hide comment
@kevin1024

kevin1024 Apr 25, 2013

I ran into the same thing. You have to set up some proxies in your karma-e2e.conf. It's kind of confusing, but you're actually running two servers:

  1. The grunt connect:test server that is started by the grunt test:e2e task. This has your app in it
  2. The karma server running on a different port. This is where your test code is executing.

Since these servers can't communicate with each other directly (they are on different ports so browser security rules apply), you need to proxy requests from server 2 to server 1. You can do this by editing the proxies array in karma-e2e.conf

Check out these files in the angular app to see how they have it set up. I used them as a template to understand how it's supposed to work:

https://github.com/angular/angular.js/blob/master/karma-e2e.conf.js
https://github.com/angular/angular.js/blob/master/Gruntfile.js

I ran into the same thing. You have to set up some proxies in your karma-e2e.conf. It's kind of confusing, but you're actually running two servers:

  1. The grunt connect:test server that is started by the grunt test:e2e task. This has your app in it
  2. The karma server running on a different port. This is where your test code is executing.

Since these servers can't communicate with each other directly (they are on different ports so browser security rules apply), you need to proxy requests from server 2 to server 1. You can do this by editing the proxies array in karma-e2e.conf

Check out these files in the angular app to see how they have it set up. I used them as a template to understand how it's supposed to work:

https://github.com/angular/angular.js/blob/master/karma-e2e.conf.js
https://github.com/angular/angular.js/blob/master/Gruntfile.js

@jacobkretz

This comment has been minimized.

Show comment
Hide comment
@jacobkretz

jacobkretz Apr 25, 2013

So no luck. Running the test below, app server start on localhost 9000; karma starts up on localhost 8080 and states that it is proxying /web to localhost 9000. when the test hits the pause() statement, I see karma running, but CurrentURL: /web points to localhost:8080 and NOT Found is listed below this. If I open another tab and go to localhost 9000 my app is there. Thanks for the help.

Karma config file:karma-e2e.conf.js

proxies = {
'/web': 'http://localhost:9000/',
};

Gruntfile.js
connect: {
options: {
port: 9000,
// Change this to '0.0.0.0' to access the server from outside.
hostname: 'localhost'
},
e2e: {
options: {
middleware: function (connect) {
return [
mountFolder(connect, '.tmp'),
mountFolder(connect, yeomanConfig.app)
];
}
}
}

karma: {

  e2e: {
    configFile: 'karma-e2e.conf.js',
    singleRun: true
  }
},

Test
'use strict';

describe('Controller: MainCtrl', function () {

it('should filter results', function() {
browser().navigateTo('/web');
//element('#sign-in').click();
pause();
expect(browser().location().url()).toBe('dashboard');
});

});

So no luck. Running the test below, app server start on localhost 9000; karma starts up on localhost 8080 and states that it is proxying /web to localhost 9000. when the test hits the pause() statement, I see karma running, but CurrentURL: /web points to localhost:8080 and NOT Found is listed below this. If I open another tab and go to localhost 9000 my app is there. Thanks for the help.

Karma config file:karma-e2e.conf.js

proxies = {
'/web': 'http://localhost:9000/',
};

Gruntfile.js
connect: {
options: {
port: 9000,
// Change this to '0.0.0.0' to access the server from outside.
hostname: 'localhost'
},
e2e: {
options: {
middleware: function (connect) {
return [
mountFolder(connect, '.tmp'),
mountFolder(connect, yeomanConfig.app)
];
}
}
}

karma: {

  e2e: {
    configFile: 'karma-e2e.conf.js',
    singleRun: true
  }
},

Test
'use strict';

describe('Controller: MainCtrl', function () {

it('should filter results', function() {
browser().navigateTo('/web');
//element('#sign-in').click();
pause();
expect(browser().location().url()).toBe('dashboard');
});

});

@kule

This comment has been minimized.

Show comment
Hide comment
@kule

kule Apr 25, 2013

Not sure if this is 100% correct but this is what we've done so far (credit @zzeniou86 for this)

Gruntfile.js

  1. change karma to include e2e as well as unit then grunt test will both unit & e2e:
    karma: {
      e2e: {
        configFile: 'karma-e2e.conf.js',
        singleRun: true
      },
      unit: {
        configFile: 'karma.conf.js',
        singleRun: true
      }
    },
  1. Change the test:server to this:
  grunt.registerTask('test', [
    'clean:server',
    'coffee',
    'compass',
    'livereload-start',
    'connect:livereload',
    'karma'
  ]);

karma-e2e.conf

  1. add the following at the bottom:
proxies =  {
    '/': 'http://localhost:9000'
  };

urlRoot = '/__e2e/';

kule commented Apr 25, 2013

Not sure if this is 100% correct but this is what we've done so far (credit @zzeniou86 for this)

Gruntfile.js

  1. change karma to include e2e as well as unit then grunt test will both unit & e2e:
    karma: {
      e2e: {
        configFile: 'karma-e2e.conf.js',
        singleRun: true
      },
      unit: {
        configFile: 'karma.conf.js',
        singleRun: true
      }
    },
  1. Change the test:server to this:
  grunt.registerTask('test', [
    'clean:server',
    'coffee',
    'compass',
    'livereload-start',
    'connect:livereload',
    'karma'
  ]);

karma-e2e.conf

  1. add the following at the bottom:
proxies =  {
    '/': 'http://localhost:9000'
  };

urlRoot = '/__e2e/';
@zzeniou86

This comment has been minimized.

Show comment
Hide comment
@zzeniou86

zzeniou86 Apr 25, 2013

Based on what @kule, the test should be updated like this:

browser().navigateTo('/');

About step (2): This is a temporary solution which starts the web application with the 'grunt server' settings. For some reason, the connect:test doesn't load the web application correctly.. which is an issue.

Based on what @kule, the test should be updated like this:

browser().navigateTo('/');

About step (2): This is a temporary solution which starts the web application with the 'grunt server' settings. For some reason, the connect:test doesn't load the web application correctly.. which is an issue.

@jacobkretz

This comment has been minimized.

Show comment
Hide comment
@jacobkretz

jacobkretz Apr 25, 2013

Success: Thanks to everyone for the help.

Success: Thanks to everyone for the help.

@jacobkretz jacobkretz closed this Apr 27, 2013

@kevin1024

This comment has been minimized.

Show comment
Hide comment
@kevin1024

kevin1024 Apr 27, 2013

I'm not so sure that this should be closed. It is at least a documentation bug since nothing mentions setting up the proxy to get e2e tests working.

I'm not so sure that this should be closed. It is at least a documentation bug since nothing mentions setting up the proxy to get e2e tests working.

@kule

This comment has been minimized.

Show comment
Hide comment
@kule

kule Apr 27, 2013

Yeah keep this open for the moment. It would be good to hear from the maintainers that we've done this correctly and like @kevin1024 said put something in the wiki/readme.

kule commented Apr 27, 2013

Yeah keep this open for the moment. It would be good to hear from the maintainers that we've done this correctly and like @kevin1024 said put something in the wiki/readme.

@jacobkretz jacobkretz reopened this Apr 27, 2013

@gaetansenn

This comment has been minimized.

Show comment
Hide comment
@gaetansenn

gaetansenn May 6, 2013

I have try to fix with your fix @kule but now the e2e karma test don't want to continue his execution :

describe('Check authentification success ', function() {

beforeEach(function () {
    browser().navigateTo('/');
});

it ('Login Test', function () {
    input('email').enter('xxxx@xxxx');
    input('password').enter('xxxx');
    element(':submit').click();
});

});

See the capture bellow :)
screen shot 2013-05-06 at 3 58 13 pm

I have try to fix with your fix @kule but now the e2e karma test don't want to continue his execution :

describe('Check authentification success ', function() {

beforeEach(function () {
    browser().navigateTo('/');
});

it ('Login Test', function () {
    input('email').enter('xxxx@xxxx');
    input('password').enter('xxxx');
    element(':submit').click();
});

});

See the capture bellow :)
screen shot 2013-05-06 at 3 58 13 pm

@gsklee

This comment has been minimized.

Show comment
Hide comment
@gsklee

gsklee May 14, 2013

Following @kule's method, it seems like you need to manually move your mouse cursor onto the elements you are testing to trigger the tests?

For example:

    it('should contain 3 items', function() {
      expect(repeater('.items li').count()).toBe(3);
    });

Then when the test runner has been fired - like @ridertahiti's screenshot - I'll have to hover onto .item to make the runner run the tests or it'll just hang there indefinitely.

gsklee commented May 14, 2013

Following @kule's method, it seems like you need to manually move your mouse cursor onto the elements you are testing to trigger the tests?

For example:

    it('should contain 3 items', function() {
      expect(repeater('.items li').count()).toBe(3);
    });

Then when the test runner has been fired - like @ridertahiti's screenshot - I'll have to hover onto .item to make the runner run the tests or it'll just hang there indefinitely.

@eddiemonge

This comment has been minimized.

Show comment
Hide comment
@eddiemonge

eddiemonge Jun 24, 2013

Member

since the only part of @kule's recommendation that is in this repo is step 3, I added that part as a PR. I commented it out so it doesnt have to be the default. #27

Member

eddiemonge commented Jun 24, 2013

since the only part of @kule's recommendation that is in this repo is step 3, I added that part as a PR. I commented it out so it doesnt have to be the default. #27

@vtanathip

This comment has been minimized.

Show comment
Hide comment
@vtanathip

vtanathip Aug 20, 2013

I have the exactly problem with @gaetansenn and @gsklee .... Don't Understand why I'm must manually trigger the test if I doesn't manually it won't execute !!!

Anyone can help with this? very confusing now.

PS: when I use new version of generator angular the problem was gone.

I have the exactly problem with @gaetansenn and @gsklee .... Don't Understand why I'm must manually trigger the test if I doesn't manually it won't execute !!!

Anyone can help with this? very confusing now.

PS: when I use new version of generator angular the problem was gone.

@amolp1709

This comment has been minimized.

Show comment
Hide comment
@amolp1709

amolp1709 Aug 29, 2013

I am facing problem of loading the jquery which i used in scenario.js file for parsing.
I am using jquery and to parse json file but chrome console showing following and my e2e test are not executing.
When I fire the command "grunt test".

Failed to load resource: the server responded with a status of 404 (Not Found) http://localhost:8080/__e2e/test/e2e/arithmatic.json
Object
scenarios.js:50
GET http://localhost:8080/base/calculator/lib/jQuery/jquery-1.10.1.min.map 404 (Not Found) :8080/base/calculator/lib/jQuery/jquery-1.10.1.min.map:1

kama-e2e.conf.js

module.exports = function(config) {
config.set({

// Karma E2E configuration

// base path, that will be used to resolve files and exclude
basePath : '../',

// list of files / patterns to load in the browser
files : [

'calculator/app/bower_components/jQuery/jquery-1.10.1.min.js',
'calculator/test/e2e/scenarios.js'

],

frameworks :["ng-scenario"],
// list of files to exclude
exclude :[],

// test results reporter to use
// possible values: dots || progress || growl
reporters: ['progress'],

// web server port
port: 8080,

// cli runner port
runnerPort: 9100,

// enable / disable colors in the output (reporters and logs)
colors :true,

// level of logging
// possible values: LOG_DISABLE || LOG_ERROR || LOG_WARN || LOG_INFO || LOG_DEBUG
//logLevel : karma.LOG_INFO,

// enable / disable watching file and executing tests whenever any file changes
autoWatch : false,

// Start these browsers, currently available:
// - Chrome
// - ChromeCanary
// - Firefox
// - Opera
// - Safari (only Mac)
// - PhantomJS
// - IE (only Windows)
browsers :['Chrome'],

// If browser does not capture in given timeout [ms], kill it
captureTimeout : 5000,

// Continuous Integration mode
// if true, it capture browsers, run tests and exit
singleRun : false,

// Uncomment the following lines if you are using grunt's server to run the tests
proxies : {
'/': 'http://localhost:9000/'
},
// URL root prevent conflicts with the site root
//urlRoot : 'karma'
//urlRoot : '/karma/'
urlRoot : '/__e2e/'
});
};

I am facing problem of loading the jquery which i used in scenario.js file for parsing.
I am using jquery and to parse json file but chrome console showing following and my e2e test are not executing.
When I fire the command "grunt test".

Failed to load resource: the server responded with a status of 404 (Not Found) http://localhost:8080/__e2e/test/e2e/arithmatic.json
Object
scenarios.js:50
GET http://localhost:8080/base/calculator/lib/jQuery/jquery-1.10.1.min.map 404 (Not Found) :8080/base/calculator/lib/jQuery/jquery-1.10.1.min.map:1

kama-e2e.conf.js

module.exports = function(config) {
config.set({

// Karma E2E configuration

// base path, that will be used to resolve files and exclude
basePath : '../',

// list of files / patterns to load in the browser
files : [

'calculator/app/bower_components/jQuery/jquery-1.10.1.min.js',
'calculator/test/e2e/scenarios.js'

],

frameworks :["ng-scenario"],
// list of files to exclude
exclude :[],

// test results reporter to use
// possible values: dots || progress || growl
reporters: ['progress'],

// web server port
port: 8080,

// cli runner port
runnerPort: 9100,

// enable / disable colors in the output (reporters and logs)
colors :true,

// level of logging
// possible values: LOG_DISABLE || LOG_ERROR || LOG_WARN || LOG_INFO || LOG_DEBUG
//logLevel : karma.LOG_INFO,

// enable / disable watching file and executing tests whenever any file changes
autoWatch : false,

// Start these browsers, currently available:
// - Chrome
// - ChromeCanary
// - Firefox
// - Opera
// - Safari (only Mac)
// - PhantomJS
// - IE (only Windows)
browsers :['Chrome'],

// If browser does not capture in given timeout [ms], kill it
captureTimeout : 5000,

// Continuous Integration mode
// if true, it capture browsers, run tests and exit
singleRun : false,

// Uncomment the following lines if you are using grunt's server to run the tests
proxies : {
'/': 'http://localhost:9000/'
},
// URL root prevent conflicts with the site root
//urlRoot : 'karma'
//urlRoot : '/karma/'
urlRoot : '/__e2e/'
});
};

@blittle

This comment has been minimized.

Show comment
Hide comment
@blittle

blittle Sep 6, 2013

@gaetansenn Did you ever figure out how to get the execution to continue?

blittle commented Sep 6, 2013

@gaetansenn Did you ever figure out how to get the execution to continue?

@amolp1709

This comment has been minimized.

Show comment
Hide comment
@amolp1709

amolp1709 Sep 6, 2013

Didn't get you ? I am able to execute the e2e test after replacing the test task by followng :
grunt.registerTask('test', [
'clean:server',
'coffee',
//'compass',
//'livereload-start',
'connect:livereload',
'karma'
]);

Didn't get you ? I am able to execute the e2e test after replacing the test task by followng :
grunt.registerTask('test', [
'clean:server',
'coffee',
//'compass',
//'livereload-start',
'connect:livereload',
'karma'
]);

@gaetansenn

This comment has been minimized.

Show comment
Hide comment
@gaetansenn

gaetansenn Sep 6, 2013

@blittle i haven't got the time to test. Keep me informed.

@blittle i haven't got the time to test. Keep me informed.

@szimek

This comment has been minimized.

Show comment
Hide comment
@szimek

szimek Oct 2, 2013

I'm trying to setup unit and e2e tests in app generated with angular generator and somehow got it working, but I'm not sure if it's correct. I've got similar config to @kule, but with a few minor differences.

Test server (connect:test) runs in app directory, but on a different port than dev server (connect:livereload):

// Gruntfile.js
connect: {
  test: {
    options: {
      middleware: function (connect) {
        return [
          mountFolder(connect, '.tmp'),
          mountFolder(connect, yeomanConfig.app) // Run in /app instead of /test directory
        ];
      },
      port: 9001 // Different port than 9000 used by dev server
    }
  }
},

karma: {
  options: {
    browsers: ['PhantomJS'],
    singleRun: true
  },
  unit: {
    configFile: 'karma.conf.js'
  },
  e2e: {
    configFile: 'karma-e2e.conf.js'
  }
}

// karma-e2e.conf.js
module.exports = function(config) {
  config.set({
    '/': 'http://localhost:9001/' // The same port as specified in Gruntfile.js for connect:test task
    urlRoot: '/_karma_/'
  });
};

When I run grunt test it runs both unit and e2e tests. I have no idea if that's the correct setup - e.g. test/runner.html file is not used at all (I think so), because connect:test doesn't run in test directory anymore. Also, I haven't figured out how to run it with singleRun: false option yet.

I'm not sure if it's an issue with generator-karma or -angular, but it would be really helpful if e2e tests worked out of the box (both single run and continuous) in an app generated with generator-angular.

szimek commented Oct 2, 2013

I'm trying to setup unit and e2e tests in app generated with angular generator and somehow got it working, but I'm not sure if it's correct. I've got similar config to @kule, but with a few minor differences.

Test server (connect:test) runs in app directory, but on a different port than dev server (connect:livereload):

// Gruntfile.js
connect: {
  test: {
    options: {
      middleware: function (connect) {
        return [
          mountFolder(connect, '.tmp'),
          mountFolder(connect, yeomanConfig.app) // Run in /app instead of /test directory
        ];
      },
      port: 9001 // Different port than 9000 used by dev server
    }
  }
},

karma: {
  options: {
    browsers: ['PhantomJS'],
    singleRun: true
  },
  unit: {
    configFile: 'karma.conf.js'
  },
  e2e: {
    configFile: 'karma-e2e.conf.js'
  }
}

// karma-e2e.conf.js
module.exports = function(config) {
  config.set({
    '/': 'http://localhost:9001/' // The same port as specified in Gruntfile.js for connect:test task
    urlRoot: '/_karma_/'
  });
};

When I run grunt test it runs both unit and e2e tests. I have no idea if that's the correct setup - e.g. test/runner.html file is not used at all (I think so), because connect:test doesn't run in test directory anymore. Also, I haven't figured out how to run it with singleRun: false option yet.

I'm not sure if it's an issue with generator-karma or -angular, but it would be really helpful if e2e tests worked out of the box (both single run and continuous) in an app generated with generator-angular.

@blittle

This comment has been minimized.

Show comment
Hide comment
@blittle

blittle Oct 2, 2013

I guess in general I am seeing the writing on the wall for the e2e testing framework. It looks as though it will be replaced with https://github.com/angular/protractor

blittle commented Oct 2, 2013

I guess in general I am seeing the writing on the wall for the e2e testing framework. It looks as though it will be replaced with https://github.com/angular/protractor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment