[Request] Add the ability to skip during test execution

[timbrinded]

Clear and concise description of the problem

Currently, describe and tests can be conditionally skipped using skipIf() properties on the top level test functions. However there are many scenarios where it is a lot cleaner, and more logical, to assess whether a test needs to be skipped only during test execution.

Suggested solution

Similar to how Mocha does this, provide test.skip() method on the global runner context which can be used to skip tests as they are being tested.

Make it so that this is callable within any part (beforeEach, beforeAll, describe, test, aftereach, afterall ) of the vitest test runner flow, and will skip that individual test or suite.

---

example:

describe("Generic Test Suite", function(){

    test("test1", function(){
    // Some state is gathered: const state
    expect(const).to.be.true
    })

    test("test2", async function(){
    // Do some lookup where value not known at initiation time
    const lookupResponse = await fetch(query)

    if (!lookupResponse){
        vi.skip()
    }
    
    expect(lookupResponse.json().field[0]).to.equal(5)

    })

})

Alternative

The skipIf() wrappers are useable but not flexible enough for many async scenarios, as they only affect whether tests get included during the gathering phase. Sometimes you may want tests to be considered skipped, instead of failed, based on responses you get.

Additional context

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[timbrinded]

Any news or comment about this one?

I've been coming across increasingly more scenarios where you would want to exit a test early and mark it as skipped, during mid execution (e.g. when asserting against prod state with expected but improper values).

[sheremet-va]

I am not against it, but it should probably be on the context, so:

test('skipped', (t) => {
  t.skip()
})

How should it be implemented - I don't know 🤷🏻 Maybe throw a specific exception that doesn't fail the test, but instead marks it as skipped?

[Aslemammad]

@sheremet-va We may need to use an abort signal maybe? In that case, each test case should be run in heavy concurrency so we'd be able to skip in any step!
Or even just mark the test as skipped and go to the next test. By throwing a specific "Symbol('skip')"

I'm just throwing ideas!

[johnkingzy]

For anyone who's looking, this is now supported

describe.skip()
it.skip()

https://vitest.dev/guide/filtering.html#skipping-suites-and-tests

[timbrinded]

@johnkingzy nope, that is not what this ticket is about. Writing your test with .skip() is not determined at execution but at prepare time.

[johnkingzy]

@timbrinded thanks for clarifying that; I haven't had to deal with this scenario before so that's good to know.

I'm just curious to know, how is using vi.skip() different from using if/else to skip running subsequent expectations based on a condition.

[timbrinded]

@johnkingzy skip if only gets checked during prepare, which is useful for disabling tests with environment variables.

It does not offer the flexibility that other test frameworks like mocha have where you can skip during test execution itself.

Any scenario that has non deterministic data in it, such as a fetch call to an API, might want to skip during certain situations. With vitest you simply have to return. Mocha will allow you to this.skip()