Since upgrading to Chrome 59 on Travis, we started seeing unit test failures due to Chrome getting disconnected from Karma. We have worked around the issue with a39bf8b and 8af203c (which cause a slight slow-down).

(For reference, we are currently using Karma v1.7.0 and jasmine v2.6.3.)

I looked into it a bit more (since this is likely to affect many people given that our setup (jasmine+Karma+Chrome) is common) and here is what I believe is happening:

The Karma server uses socket.io to (among other things) get updates from the browser about the test results. Whenever the browser reports a test result, it refreshes the noActivityTimer, but if it doesn't hear from the browser for 10s it considers it disconnected.

For whatever reason, socket.io uses it's XHR-based polling implementation instead of WebSocket (but the problem would arise with WebSocket too). When the socket wants to emit a result, it will (after some hoops) call its flush() method, which will send the message, unless its Transport (i.e. the polling thingy) is currently not writable.

So, what causes the Transport to not be writable?

• transport.send() calls transport.write()
• transport.write() sets transport.writable = false, creates a callback for setting it back to true, and passes the callback to transport.doWrite().
  Note: Until the callback is called, transport.writable will be false and the socket won't send anything to the Karma server.
• transport.doWrite() calls [transport.request()][17] (which essentially [creates][18] and [sends][19] an XMLHttpRequest) and binds the callback to the request's success event (long story short to the XHR's [.onreadystatechangehandler][20] (forreadyState === 4`)).

What this all means is that once the socket sends a message to the server, it has to wait for the XHR to complete before sending another message.

In the meantime, in Jasmine-land:
Jasmine is running the tests using its QueueRunner. Once it is gone through the whole queue (synchronously in our case), it will call its clearStack() method (and then continue running tests). The clearStack() method is what is passed to the QueueRunner, which (in a nutshell) is based on MessageChannel.

Let's tie everything together :)

Jasmine runs some tests. Clears the stack. The socket sends a message to inform the Karma server about the test result. Jasmine runs more tests. Etc.

Now, if everything ran synchronously, the readystatechange event listener for the polling socket doesn't get a chance to fire and set Transport.writable to true, in order to allow more messages to be sent to the Karma server. Our tests do run synchronously, but what about the clearStack() method? It is not, but...

It turns out that MessageChannel.postMessage() uses a task queue that gets processed before the one XHR events use. As a result, Jasmine will keep running tests and clearing the stack and running more tests, before the first XHR has a chance to complete. Thus, the socket's Transport will be non-writable all the time and Karma will not receive any updates from the browser.

Why didn't it fail before?

It seems that there were some changes in Chrome 58 or 59 wrt .postMessage() (these two seem related, but are also kind of old 😕 : 04055dd, 05a1330). I.e. previously it would use the Timer queue (what setTimeout uses), while now it uses a different queue that is processed before the Timer queue. Consider the following example:

function test() {
  // XHR.onload
  const xhr = new XMLHttpRequest();
  xhr.open('GET', location.href);
  xhr.onload = () => console.log('load');
  xhr.send();   // Assume it completes under 5s.

  // setTimeout
  setTimeout(() => console.log('timeout'));

  // requestAnimationFrame
  requestAnimationFrame(() => console.log('raf'));

  // MessageChannel
  const ch = new MessageChannel();
  const start = Date.now();
  ch.port1.onmessage = () => {
    console.log('channel-XHR.readyState:', xhr.readyState);
    for (let i = 0; i < 1000; ++i) {}
    if (Date.now() - start < 5000) ch.port2.postMessage('bar');   // Run for 5s.
  };
  ch.port2.postMessage('bar');

  // Promise
  Promise.resolve().then(() => console.log('promise'));
}

The output on Chrome 57 is different than on Chrome 59:

Chrome 57:

promise
raf
load
timeout
channel-XHR.readyState: 4 (many times)

Chrome 59:

promise
channel-XHR.readyState: 1 (many times)
raf
timeout
load

Why does beforeEach(done => done()) work?

By having an argument, the callback takes the async path. But because done() is called synchronously, it will be called before completedAsynchronously is set to false, which will in turn run the rest of the queued callbacks in a setTimeout(), giving the XHR time to complete.