expectTypeOf - Tests passes even though there are errors

[ivodolenc]

Describe the bug

Hi, first of all thanks for your work on this awesome project.

I would like to use Vitest for type checking.

I came across this example in the Vitest documentation, but it doesn't seem to work as it should.

The test passes even though there is a type checking error.

Is it intentional behavior? I expected the test to fail and report an error.

Reproduction

package.json

{
  "scripts": {
    "test:types": "vitest typecheck"
  },
  "devDependencies": {
    "vitest": "^0.31.1"
  }
}

tsconfig.json

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "Node",
    "skipLibCheck": true,
    "strict": true,
    "allowJs": true,
    "noEmit": true,
    "resolveJsonModule": true,
    "allowSyntheticDefaultImports": true,
    "forceConsistentCasingInFileNames": true
  },
  "include": ["src"],
  "exclude": ["node_modules", "dist"]
}

vitest.config.ts

import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    typecheck: {
      include: ['test/types/**/*.ts']
    }
  }
})

test/types/test-example.ts

// test/types/test-example.ts

import { test, expectTypeOf } from 'vitest'

test('test-example', () => {
  expectTypeOf({ a: 1 }).toEqualTypeOf<{ a: boolean }>() // It reports an error, but the test passes
})

console report

 ✓ test/types/test-example.ts

Test Files  1 passed (1)
     Tests  1 passed (1)
Type Errors  no errors
     Start at  15:47:32
     Duration  1.02s

PASS  Waiting for file changes...
      press q to quit

System Info

- mac: ventura latest
- chrome: latest 
- npm: 9.6.4 
- node: 18.12.1 
- vitest: 0.31.1

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[ivodolenc]

Also, watch mode doesn't seem to register changes to the file, i.e. it doesn't reload onSave.

It reports in the console PASS Waiting for file change... but nothing happens when I make changes and save the file.

[ivodolenc]

I'm not sure as I only did quick tests, but deep nesting also seems to pass the tests for different types although it should throw an error.

Passes

This passes the test, but it shouldn't.

import { expectTypeOf } from 'vitest'

type Test = {
  object: {
    object: {
      number: number
      string: string
      boolean: boolean
      object: {
        number: number
        string: string
        boolean: boolean
      }
    }
  }
}

type TestNotEqual = {
  object: {
    object: {
      number?: number
      object?: {
        number?: number
      }
    }
  }
}

expectTypeOf<Test>().toEqualTypeOf<TestNotEqual>() // this passes the test and does not report an error

Not Passing

This doesn't pass the test, but that's expected.

As if it recursively checks only the first or second level?

import { expectTypeOf } from 'vitest'

type Test = {
  object: {
    string: string
    object: {
      object: {
        number: number
      }
    }
  }
}

type TestNotEqual = {
  object: {
    number: number // registers the number as an error since it should be a string, which is expected
    object: {
      object: {
        number: number
      }
    }
  }
}

expectTypeOf<Test>().toEqualTypeOf<TestNotEqual>() // this fails the test and throws an error

[sheremet-va]

Your tsconfig doesn't include "test" folder:

-  "include": ["src"],
+  "include": ["src", "test"],

[sheremet-va]

It should show an error that no files were found though. tsc exits with exitCode 1

[ivodolenc]

Hey @sheremet-va, yes you are right, it works as expected after I added "include": ["src", "test"] to the tsconfig.json.

Thanks for pointing that out...

Also, watch works as expected, reloads fine on file change.

But it seems that this still have some issues. Should I open a new issue for that specifically?

As far as I'm concerned, the rest of this works as expected so you can close as resolved.

[sheremet-va]

But it seems that this still have some issues. Should I open a new issue for that specifically?

expectTypeOf comes from this package: https://github.com/mmkal/expect-type#readme

You can open an issue there

[ivodolenc]

expectTypeOf comes from this package: https://github.com/mmkal/expect-type#readme

Yes, I assumed. I reported a bug there too.

Just a friendly note for others, expectTypeOf doesn't seem to work recursively as expected so keep an eye on that.

Okay, feel free to close this as resolved. Thanks for your time and help!

[ivodolenc]

But it seems that this still have some issues. Should I open a new issue for that specifically?

expectTypeOf comes from this package: https://github.com/mmkal/expect-type#readme

You can open an issue there

Hey @sheremet-va, just a friendly reminder, this has been fixed in the latest version v0.16.0. See comment.