Tests not running headlessly with webpack

[gavmor]

Hey, team!

Is there a known issue with headless specs & webpack, or should I impugn my setup?

Tossing the example code into a gulp file worked out great:

gulp.task('test-browser', function() {
  return gulp.src(['test/**/*_test.js'])
    .pipe(webpack(webpackConfig))
    .pipe(jasmineBrowser.specRunner())
    .pipe(jasmineBrowser.server({whenReady: plugin.whenReady}));
});

But we ran into problems when we tried to run specs headlessly, i.e.:

gulp.task('test', function() {
  return gulp.src(['test/**/*_test.js'])
    .pipe(webpack(webpackConfig))
    .pipe(jasmineBrowser.specRunner({console: true}))
    .pipe(jasmineBrowser.headless({whenReady: plugin.whenReady}));
});

The above task logged changes, but didn't seem to run tests:

[11:42:41] Starting 'test'...
[11:42:44] Version: webpack 1.12.9
  Asset     Size  Chunks             Chunk Names
spec.js  1.03 MB       0  [emitted]  main
[11:42:44] webpack is watching for changes
[11:42:46] Version: webpack 1.12.9
[11:42:46] webpack is watching for changes
[11:42:47] Version: webpack 1.12.9
[11:42:47] webpack is watching for changes
[11:42:48] Version: webpack 1.12.9
[11:42:48] webpack is watching for changes
[11:42:49] Version: webpack 1.12.9
[11:42:49] webpack is watching for changes

Weirdly, taking 'the watch' out of our webpack config resulted in this:

[11:42:28] Starting 'test'...
[11:42:31] Version: webpack 1.12.9
  Asset     Size  Chunks             Chunk Names
spec.js  1.03 MB       0  [emitted]  main
[11:42:31] Jasmine server listening on port 8000
TypeError: 'undefined' is not a function (evaluating 'RegExp.prototype.test.bind(/^(data|aria)-[a-z_][a-z\d_.\-]*$/)')

  http://localhost:8000/spec.js:10424
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:8526
  http://localhost:8000/spec.js:8604
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:186
  http://localhost:8000/spec.js:261
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:137
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:117
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:93
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:58
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:47
  http://localhost:8000/spec.js:20 in __webpack_require__
  http://localhost:8000/spec.js:40
  http://localhost:8000/spec.js:26901

No specs found
Finished in 0.001 seconds
[11:42:33] Finished 'test' after 4.21 s

When we removed webpack entirely, we got the expected SyntaxError, but also "No Specs found":

11:37:30] Starting 'test'...
[11:37:30] Jasmine server listening on port 8000
SyntaxError: Parse error

SyntaxError: Parse error

SyntaxError: Parse error


No specs found
Finished in 0.001 seconds
[11:37:31] Finished 'test' after 1.2 s

[charleshansen]

It looks like you are using an old version of phantomjs. If you are using
phantom <2, you often need to include the bind polyfill. It is easiest to
just use phantom 2. You can't use npm to install phantom 2, but brew works.

Anyway, once phantomjs is sorted out, headless mode should work. You
probably do not want the 'watch' in webpack config for headless.
On Dec 23, 2015 1:31 PM, "Gavin Morgan" notifications@github.com wrote:

Hey, team!

Is there a known issue with headless specs & webpack, or should I impugn
my setup?

Tossing the example code into a gulp file worked out great:

gulp.task('test-browser', function() {
return gulp.src(['test/*_/__test.js'])
.pipe(webpack(webpackConfig))
.pipe(jasmineBrowser.specRunner())
.pipe(jasmineBrowser.server({whenReady: plugin.whenReady}));
});

But we ran into problems when we tried to run specs headlessly, i.e.:

gulp.task('test', function() {
return gulp.src(['test/*_/__test.js'])
.pipe(webpack(webpackConfig))
.pipe(jasmineBrowser.specRunner({console: true}))
.pipe(jasmineBrowser.headless({whenReady: plugin.whenReady}));
});

The above task logged changes, but didn't seem to run tests:

[11:42:41] Starting 'test'...
[11:42:44] Version: webpack 1.12.9
Asset Size Chunks Chunk Names
spec.js 1.03 MB 0 [emitted] main
[11:42:44] webpack is watching for changes
[11:42:46] Version: webpack 1.12.9
[11:42:46] webpack is watching for changes
[11:42:47] Version: webpack 1.12.9
[11:42:47] webpack is watching for changes
[11:42:48] Version: webpack 1.12.9
[11:42:48] webpack is watching for changes
[11:42:49] Version: webpack 1.12.9
[11:42:49] webpack is watching for changes

Weirdly, taking 'the watch' out of our webpack config resulted in this:

[11:42:28] Starting 'test'...
[11:42:31] Version: webpack 1.12.9
Asset Size Chunks Chunk Names
spec.js 1.03 MB 0 [emitted] main
[11:42:31] Jasmine server listening on port 8000
TypeError: 'undefined' is not a function (evaluating 'RegExp.prototype.test.bind(/^(data|aria)-[a-z_][a-z\d_.-]*$/)')

http://localhost:8000/spec.js:10424
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:8526
http://localhost:8000/spec.js:8604
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:186
http://localhost:8000/spec.js:261
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:137
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:117
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:93
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:58
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:47
http://localhost:8000/spec.js:20 in webpack_require
http://localhost:8000/spec.js:40
http://localhost:8000/spec.js:26901

No specs found
Finished in 0.001 seconds
[11:42:33] Finished 'test' after 4.21 s

When we removed webpack entirely, we got the expected SyntaxError, but
also "No Specs found":

11:37:30] Starting 'test'...
[11:37:30] Jasmine server listening on port 8000
SyntaxError: Parse error

SyntaxError: Parse error

SyntaxError: Parse error

No specs found
Finished in 0.001 seconds
[11:37:31] Finished 'test' after 1.2 s

—
Reply to this email directly or view it on GitHub
#22.

[gavmor]

Fantastic! Upgrading to 2.0 worked like a charm without watch: true, but I actually do want a task that watches and runs specs headlessly, which remains obstinately quiet.

Here's a tiny repo with three gulp tasks: spec, spec-console, & spec-console-watch.

After npm install, you can see gulp spec-console fail tests, and gulp spec-console-watch log recognition of changes without re-running specs.

[rdy]

Gulp jasmine browser expects the stream to be ended in order to signal the creation of the test suite. We would need some other mechanism to support some kind of auto-testing the way you are describing. I'm not sure what we could use to signal the test runner to start creating the test suite and running it.

I think the better solution would be to create a watching task that re-runs the task when it detects a change:

gulp.task('spec-console-watch', function() {
  gulp.watch(specFiles, ['spec-console']);
});

I think this would accomplish what you want, it would automatically re-run the spec-console task whenever any spec file changes. It is more idiomatic gulp as well. The only downside to this approach is that webpack wouldn't re-use the partially built files between changes. It would probably still work decently though, your thoughts?

[rdy]

I believe that using the watch task in the way I've specified should fix the issue, please re-open this issue if it is still a problem.

[gavmor]

Your suggestion was golden, @rdy! I am grateful for both your and @charleshansen's contributions.
This project has enabled a fun & stable JS environment!

[avitevet]

Thanks @rdy , I was able to use gulp+jasmine+slimerjs with watching on Win10 x64 using that method.

[jgonggrijp]

In response to @rdy a few posts above: I arrived at this solution, too, although with Browserify+Watchify instead of WebPack. I don't find the solution very satisfying, because it means creating a new Jasmine server and firing up a new PhantomJS instance every time any of my sources changes. Or at least, that's my impression of how gulp-jasmine browser works. Full story on StackOverflow.

I'd like to keep the Jasmine server and the PhantomJS instance between changes. Is there anything I can do to hack around the specrunner's requirement to receive the end event? Can I somehow listen to watchify's update, manually add an end event to the pipe and re-open the headless server for new input after every end?

[rdy]

@jgonggrijp if you start an instance of the jasmine server and run it on the same port by specifying that port in the options it does not require you to start one up again. This is documented in the README.
The requirement for this is that you specify a PORT when starting up the server, or running an existing one.

If you don't already have a jasmine server running it will start one up on the fly. I haven't tried the browserify + watchify combination myself so I can't comment on its efficacy.

As for the end event, it is a requirement of jasmine runner itself. If you wanted a workaround you'd need to make an issue on the jasmine github issues.

[jgonggrijp]

@rdy

if you start an instance of the jasmine server and run it on the same port by specifying that port in the options it does not require you to start one up again.

Thanks, this is a relief. So for maximum clarity: is it enough to just specify the port when calling .headless, or should I also manually start a Jasmine server on the same port in advance?

Will this also preserve the driver instance?

This is documented in the README.

Sorry, I can't find the place in the README where this is explained. Could you quote the relevant part?

[rdy]

@jgonggrijp you can specify the port when calling headless, if you give it 8888 for example it will re-use the server (if you have one running) otherwise it will start one up from scratch. If you don't give headless a port option it will always try to find a new port and start up the server.

If you don't want to use the DEFAULT_JASMINE_PORT of 8888 you can specify the port to use in the server options as {port: 8080} for example.