Skip to content

New testing framework #7151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 59 commits into
base: main
Choose a base branch
from
Open

Conversation

timotheeguerin
Copy link
Member

@timotheeguerin timotheeguerin commented Apr 28, 2025

Tentative improvement of the testing framework for TypeSpec using the idea here https://gist.github.com/timotheeguerin/e218fd2079fb331db31bc0a5a70dc539

Todo:

  • Add tests
  • Figure out the program on the static function
  • Better type collection
  • Add doc comments on all the types
  • Check experiences for emitters(calling $onEmit)
  • Double check the async fs contruction if we can have a better pattern?
  • Generalize Filesystem(add raw file vs js file)
  • Emitter tester should have the same pipe functions as the base tester

Copy link
Contributor

github-actions bot commented Apr 28, 2025

All changed packages have been documented.

  • @typespec/compiler
  • @typespec/http
  • @typespec/openapi
  • @typespec/openapi3
  • @typespec/rest
  • @typespec/sse
  • @typespec/streams
  • @typespec/versioning
  • @typespec/xml
Show changes

@typespec/compiler - feature ✏️

[API] Addition of a new testing framework. See https://typespec.io/docs/extending-typespec/testing

@typespec/http - internal ✏️

Migrated to new tester

@typespec/openapi - internal ✏️

Migrated to new tester

@typespec/openapi3 - internal ✏️

Migrated to new tester

@typespec/rest - internal ✏️

Migrated to new tester

@typespec/sse - internal ✏️

Migrated to new tester

@typespec/streams - internal ✏️

Migrated to new tester

@typespec/versioning - internal ✏️

Migrated to new tester

@typespec/xml - internal ✏️

Migrated to new tester

@azure-sdk
Copy link
Collaborator

azure-sdk commented Apr 28, 2025

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

@timotheeguerin timotheeguerin changed the title [WIP] Tester v2 New testing framework Jun 3, 2025
@timotheeguerin timotheeguerin marked this pull request as ready for review June 3, 2025 16:29
```ts title="test/tester.ts"
import { createTester } from "@typespec/compiler/testing";

const MyTester = createTester({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be helpful to mention importLibraries() needs to be called to actually import these in a program, given the older test wrapper did this automatically.

strictEqual(Foo.name, "Foo");
```

2. Using flourslash syntax to mark the types you want to collect (`/*foo*/`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows you to get types that can't be decorated, right? Maybe worth calling that out (otherwise it's not clear why you'd use this over @test)

If you are using VS Code, you can install the [Node test runner](https://marketplace.visualstudio.com/items?itemName=connor4312.nodejs-testing) to run your tests from the editor. This will also allow you to easily debug your tests.

After installing the extension, you should be able to discover, run, and debug your tests from the test explorer.
[Testing](./testing.mdx) see documentation for adding tests to your library.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A migration guide might be useful (or even point back to this PR so someone see the changes that need to be made)


@test op ${t.op("get")}(): EmbeddingVector<int32>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: We don't need the @test decorators since we're using t.<entity> - but cool to see you can use them both together and nothing bad happens! (Looks like @test is still used a lot in this file)

Co-authored-by: Christopher Radek <14189820+chrisradek@users.noreply.github.com>

export interface TestCompileOptions {
/** Optional compiler options */
readonly options?: CompilerOptions;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to compileOptions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants