Using assertion outside the test context leads to nasty crash without good message

[bahmutov]

If I use "expect" outside a describe block, Cypress 0.18.1 fails to parse the file and shows "no tests found message". It is very unclear to the user why the tests do not start.

It could also related to the "cy.log" command I have added by using #342
I could not reliably recreate the crash in the self-contained example, but this is the principle

function foo(name) {
  expect(name).to.be.a('string')
}
describe('foo', () => {
  it('works', () => {
    foo('foo')
  })
})

Trying to run this breaks Cypress with message "No tests found in your file:" and the following error stack and object arguments (this shows that this is the "expect" assertion crashing)

[bahmutov]

Even a "simple" exception is not caught in the spec.js file, causing confusion. For example

// example_spec.js
throw new Error('nope')
describe('Kitchen Sink', function(){ ...

shows

At least should advise the user to open dev tools? Or open dev tools automatically...

[bahmutov]

Also, it seems this error is linked to https://docs.cypress.io/docs/errors#section-no-tests-found-in-your-file while it is more appropriate to link to https://docs.cypress.io/docs/errors#section-we-found-an-error-preparing-your-test-file

[brian-mann]

This is simply an uncaught global exception (which we are not correctly catching) from the spec file. We'll catch errors obviously from within a test but not outside of it.

There's a difference between parse errors and runtime errors. Runtime errors simply lead Cypress to not find any valid tests (since the code bails early) - whereas parse / bundle errors are caught prior to sending down the spec files and can be displayed.

The solution here is simple - catch global uncaught spec errors so we can notify the user there's something wrong with their code.

We already catch global uncaught errors from the application under test, so that same code can be repurposed to catch spec errors.

[bahmutov]

agree this is the best long term solution. Double points because it solves problem a new user might have (which are higher priority in my mind)

[brian-mann]

The code for this is done, but this has yet to be released. We'll update the issue and reference the changelog when it's released.