Sometimes tests do not start

[bahmutov]

Hi,

When running CI we periodically get the following cryptic error

Tests should begin momentarily...

Error processing argument at index 0, conversion failure from null
TypeError: Error processing argument at index 0, conversion failure from null
  at TypeError (native)
  at exit (/root/.cypress/Cypress/resources/app/index.js:1752:20)
  at tryCatcher (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/util.js:16:23)
  at Promise._settlePromiseFromHandler (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:503:31)
  at Promise._settlePromise (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:560:18)
  at Promise._settlePromise0 (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:605:10)
  at Promise._settlePromises (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:684:18)
  at Async._drainQueue (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:126:16)
  at Async._drainQueues (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:136:10)
  at Immediate.Async.drainQueues [as _onImmediate] (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:16:14)
  at tryOnImmediate (timers.js:534:15)
  at processImmediate [as _immediateCallback] (timers.js:514:5)

I believe that this has to do with some bootstrapping issue?

[brian-mann]

Are you on the latest version 0.16.4 ?

[bahmutov]

oh yes

[bahmutov]

I also saw this failure which is rare - it ran one test, but then failed

Starting tests at Fri Jul 15 19:41:26 UTC 2016
$ cypress ci --spec cypress/integration/$CI_BUILD_NAME.js

Tests should begin momentarily...



  Knowledge Base Published

    ✓ loads (4069ms)
Error processing argument at index 0, conversion failure from null
TypeError: Error processing argument at index 0, conversion failure from null
  at TypeError (native)
  at exit (/root/.cypress/Cypress/resources/app/index.js:1752:20)
  at tryCatcher (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/util.js:16:23)
  at Promise._settlePromiseFromHandler (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:503:31)
  at Promise._settlePromise (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:560:18)
  at Promise._settlePromise0 (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:605:10)
  at Promise._settlePromises (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:684:18)
  at Async._drainQueue (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:126:16)
  at Async._drainQueues (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:136:10)
  at Immediate.Async.drainQueues [as _onImmediate] (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:16:14)
  at tryOnImmediate (timers.js:534:15)
  at processImmediate [as _immediateCallback] (timers.js:514:5)

[shauvik]

We are also getting this error. We had disabled a test last week which was failing on this error but this week another test fails on this error. Any insights on what is going wrong?

Error processing argument at index 0, conversion failure from null
TypeError: Error processing argument at index 0, conversion failure from null
  at TypeError (native)
  at exit (/root/.cypress/Cypress/resources/app/index.js:1752:20)
  at tryCatcher (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/util.js:16:23)
  at Promise._settlePromiseFromHandler (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:503:31)
  at Promise._settlePromise (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:560:18)
  at Promise._settlePromise0 (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:605:10)
  at Promise._settlePromises (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/promise.js:684:18)
  at Async._drainQueue (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:126:16)
  at Async._drainQueues (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:136:10)
  at Immediate.Async.drainQueues [as _onImmediate] (/root/.cypress/Cypress/resources/app/node_modules/bluebird/js/release/async.js:16:14)
  at tryOnImmediate (timers.js:534:15)
  at processImmediate [as _immediateCallback] (timers.js:514:5)

[brian-mann]

Okay so this error happens due to a null value being passed to process.exit.

We process.exit with the number of test failures so that the calling process in CI knows whether or not the CI run failed or not.

In Linux we have to re-spawn several processes and chain them all together and this is likely where the problem is. When the deepest process exits, it needs to pass up its exit code so the parent proces also exits with the same exit code. In other words we have to juggle from child->parent all the way up.

What doesn't make sense about this is for it to be intermittent. We have literally thousands of CI runs across dozens of projects per week (all in Linux) and have never seen this error. I would assume any incorrect exiting logic would be consistent, and either error or not every single time.

With that said, it's fairly easy to coerce null values into 0 but I feel this may be masking a potentially different issue. There should never be a null value being passed up. Nevertheless, we will fix this in 0.17.0 and if for whatever reason it's not ready this weekend we'll release 0.16.5 with this fix this weekend.

[brian-mann]

Fixed in 0.16.5.

In the latest released we are now coercing null values into 0 but please monitor this and if you find issues where exit codes are incorrect (such as tests failing but exiting with 0 or vise versa) let us know.

It shouldn't happen and we have lots of tests around this, but I was never ever to determine exactly how a null value could be passed to process.exit.