Skip to content

New testing framework #7151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 62 commits into from
Jun 24, 2025
Merged
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
dec22bd
Initial host builder
timotheeguerin Apr 26, 2025
9a669a7
wrap
timotheeguerin Apr 27, 2025
69b55ff
Clone fs
timotheeguerin Apr 27, 2025
4452fcd
cleanup
timotheeguerin Apr 27, 2025
97e51a7
tweaks
timotheeguerin Apr 28, 2025
a534b40
auto import and using
timotheeguerin Apr 28, 2025
48fc2c2
.
timotheeguerin Apr 28, 2025
8242a64
Simplify
timotheeguerin Apr 28, 2025
4af3ea0
marked template v1
timotheeguerin Apr 29, 2025
5d22b31
Simpler?
timotheeguerin Apr 29, 2025
7b57e21
Merge branch 'main' of https://github.com/Microsoft/typespec into tes…
timotheeguerin May 9, 2025
219a2d9
connect marker
timotheeguerin May 9, 2025
a6885a4
works with values
timotheeguerin May 9, 2025
dcf71e2
Better
timotheeguerin May 9, 2025
595ff1b
.
timotheeguerin May 9, 2025
fd4c0ce
type check
timotheeguerin May 9, 2025
e01aec9
update openapi
timotheeguerin May 9, 2025
59779c7
remove log
timotheeguerin May 9, 2025
555512a
.
timotheeguerin May 9, 2025
e0dd02b
simplify
timotheeguerin May 9, 2025
ccc79ca
test
timotheeguerin May 9, 2025
9e16b20
.
timotheeguerin May 9, 2025
ab0380e
.
timotheeguerin May 9, 2025
342c388
Works with multi file
timotheeguerin May 9, 2025
ad30cbf
fix
timotheeguerin May 9, 2025
e8b3792
simplify
timotheeguerin May 11, 2025
776bf33
emitter testing support
timotheeguerin May 12, 2025
f2d9522
Merge branch 'main' of https://github.com/Microsoft/typespec into tes…
timotheeguerin May 13, 2025
9205a98
Separate FS and add generic functions
timotheeguerin May 13, 2025
afc3463
try
timotheeguerin May 13, 2025
f287465
Create tester
timotheeguerin May 13, 2025
a64c1e0
Missing
timotheeguerin May 13, 2025
9f3c4a3
fix
timotheeguerin May 13, 2025
12402d2
fixup and basic connect to openapi3
timotheeguerin May 13, 2025
37d93a2
More migration openapi3
timotheeguerin May 13, 2025
b512337
more fixes
timotheeguerin May 13, 2025
a306d02
Simplify openapiFor
timotheeguerin May 13, 2025
d342f06
fix
timotheeguerin May 14, 2025
627d219
Migrate xml
timotheeguerin May 14, 2025
8acb8dd
Migrate sse and streams
timotheeguerin May 14, 2025
61f8c52
migrate rest
timotheeguerin May 14, 2025
600686c
migrate http and better node resolution
timotheeguerin May 14, 2025
d9bdc80
migrate versioning
timotheeguerin May 14, 2025
8f941cb
Handle change to output dir
timotheeguerin May 14, 2025
d8d3596
Merge branch 'main' into tester-v2
timotheeguerin May 14, 2025
4d7ecd3
Create tester-v2-2025-4-14-20-23-17.md
timotheeguerin May 15, 2025
36814a8
Create tester-v2-2025-4-15-17-52-11.md
timotheeguerin May 15, 2025
7ca1450
Merge branch 'main' of https://github.com/Microsoft/typespec into tes…
timotheeguerin May 23, 2025
9acf7db
Rename
timotheeguerin May 23, 2025
92c65ab
parity and add files
timotheeguerin May 23, 2025
3b379e2
piping
timotheeguerin May 23, 2025
7c8a7ec
Fix values
timotheeguerin May 23, 2025
79a1e38
works
timotheeguerin May 23, 2025
ac12908
Merge branch 'main' into tester-v2
timotheeguerin May 29, 2025
e19dbaa
Merge branch 'main' into tester-v2
timotheeguerin Jun 3, 2025
c59c9ec
add docs for marked template
timotheeguerin Jun 3, 2025
6f6fd7a
add docs
timotheeguerin Jun 3, 2025
d96573f
add back for back compat
timotheeguerin Jun 3, 2025
1db6ad0
Apply suggestions from code review
timotheeguerin Jun 9, 2025
46e607e
Merge branch 'main' of https://github.com/Microsoft/typespec into tes…
timotheeguerin Jun 21, 2025
729e5eb
handle review comments
timotheeguerin Jun 21, 2025
983c2e4
.
timotheeguerin Jun 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Initial host builder
  • Loading branch information
timotheeguerin committed Apr 26, 2025
commit dec22bd6ef6aa5e49a98aa7e9039f6412517b69a
3 changes: 3 additions & 0 deletions packages/compiler/src/testing/index.ts
Original file line number Diff line number Diff line change
@@ -34,3 +34,6 @@ export type {
TypeSpecTestLibrary,
TypeSpecTestLibraryInit,
} from "./types.js";

// TODO: use named imports
export * from "./test-host-v2.js";
129 changes: 129 additions & 0 deletions packages/compiler/src/testing/test-host-v2.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
import {
CompilerOptions,
Diagnostic,
NoTarget,
NodeHost,
Program,
SourceFile,
Type,
compile,
compilerAssert,
getRelativePathFromDirectory,
joinPaths,
resolvePath,
} from "@typespec/compiler";
import { readFile } from "fs/promises";
import { createSourceLoader } from "../core/source-loader.js";
import { StandardTestLibrary, createTestFileSystem } from "./test-host.js";
import { resolveVirtualPath } from "./test-utils.js";

export interface TestCompileResult {
readonly program: Program;
readonly types: Record<string, Type>;
}

export interface JsFileDef {
[key: string]: string | unknown;
}

interface TestCompileOptions {
readonly files?: Record<string, string | JsFileDef>;
readonly options?: CompilerOptions;
}

interface Testable {
// compile(
// main: string,
// options?: TestCompileOptions
// ): Promise<TestCompileResult>;
// diagnose(
// main: string,
// options?: TestCompileOptions
// ): Promise<readonly Diagnostic[]>;
compileAndDiagnose(
main: string,
options?: TestCompileOptions,
): Promise<[TestCompileResult, readonly Diagnostic[]]>;
}

// Immutable structure meant to be reused
export interface TestHostBuilder extends Testable {
// addImports(): TestHostBuilder;
// addUsing(...names: string[]): TestHostBuilder;
// wrap(fn: (x: string) => string): TestHostBuilder;
// createHost(): TestHostV2;
}

export function createTestHostBuilder(
base: string,
options: { libraries: string[] },
): TestHostBuilder {
let loaded: Promise<void> | undefined;
const fs = createTestFileSystem();
fs.addTypeSpecFile(".keep", ""); // dummy so it knows / is a directory TODO: better way to do this?

return {
compileAndDiagnose,
};

function load(): Promise<void> {
if (loaded) return loaded;

loaded = loadInternal();
return loaded;

async function loadInternal() {
const sl = await createSourceLoader({ ...NodeHost, realpath: async (x) => x });
const selfName = JSON.parse(await readFile(resolvePath(base, "package.json"), "utf8")).name;
for (const lib of options.libraries) {
await sl.importPath(lib, NoTarget, base);
}

await fs.addTypeSpecLibrary(StandardTestLibrary);
fs.addTypeSpecFile(".tsp/test-lib/main.tsp", 'import "./test.js";');
fs.addJsFile(".tsp/test-lib/test.js", {
namespace: "TypeSpec",
$test(_: any, target: Type) {},
});

function computeVirtualPath(file: SourceFile): string {
const context = sl.resolution.locationContexts.get(file);
compilerAssert(
context?.type === "library",
`Unexpected: all source files should be in a library but ${file.path} was in '${context?.type}'`,
);
const relativePath = getRelativePathFromDirectory(base, file.path, false);
if (context.metadata.name === selfName) {
return joinPaths("node_modules", selfName, relativePath);
} else {
return relativePath;
}
}

for (const file of sl.resolution.sourceFiles.values()) {
const relativePath = computeVirtualPath(file.file);
fs.addTypeSpecFile(resolveVirtualPath(relativePath), file.file.text);
}
for (const file of sl.resolution.jsSourceFiles.values()) {
const relativePath = computeVirtualPath(file.file);
fs.addJsFile(resolveVirtualPath(relativePath), file.esmExports);
}
for (const [path, lib] of sl.resolution.loadedLibraries) {
fs.addTypeSpecFile(
resolvePath("node_modules", path, "package.json"),
(lib.manifest as any).file.text,
);
}
}
}

async function compileAndDiagnose(
main: string,
options?: TestCompileOptions,
): Promise<[TestCompileResult, readonly Diagnostic[]]> {
await load();
fs.addTypeSpecFile("main.tsp", main);
const program = await compile(fs.compilerHost, resolveVirtualPath("main.tsp"));
return [{ program, types: {} }, program.diagnostics];
}
}
2 changes: 1 addition & 1 deletion packages/compiler/src/testing/test-host.ts
Original file line number Diff line number Diff line change
@@ -152,7 +152,7 @@ function createTestCompilerHost(
};
}

export async function createTestFileSystem(options?: TestHostOptions): Promise<TestFileSystem> {
export function createTestFileSystem(options?: TestHostOptions): TestFileSystem {
const virtualFs = createStringMap<string>(!!options?.caseInsensitiveFileSystem);
const jsImports = createStringMap<Promise<any>>(!!options?.caseInsensitiveFileSystem);

1 change: 1 addition & 0 deletions packages/openapi/test/helpers.test.ts
Original file line number Diff line number Diff line change
@@ -3,6 +3,7 @@ import { BasicTestRunner, createTestRunner } from "@typespec/compiler/testing";
import { strictEqual } from "assert";
import { beforeEach, describe, it } from "vitest";
import { resolveOperationId } from "../src/helpers.js";

describe("openapi: helpers", () => {
let runner: BasicTestRunner;

11 changes: 10 additions & 1 deletion packages/openapi/test/test-host.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
import { createTestHost, createTestWrapper } from "@typespec/compiler/testing";
import { resolvePath } from "@typespec/compiler";
import {
createTestHost,
createTestHostBuilder,
createTestWrapper,
} from "@typespec/compiler/testing";
import { HttpTestLibrary } from "@typespec/http/testing";
import { RestTestLibrary } from "@typespec/rest/testing";
import { OpenAPITestLibrary } from "../src/testing/index.js";

export const HostBuilder = createTestHostBuilder(resolvePath(import.meta.dirname, ".."), {
libraries: ["@typespec/http", "@typespec/rest", "@typespec/openapi"],
});

export async function createOpenAPITestHost() {
return createTestHost({
libraries: [HttpTestLibrary, RestTestLibrary, OpenAPITestLibrary],
17 changes: 17 additions & 0 deletions packages/openapi/test/test-new-host.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { expectDiagnostics } from "@typespec/compiler/testing";
import { it } from "vitest";
import { HostBuilder } from "./test-host.js";

it("works", async () => {
const [, diagnostics] = await HostBuilder.compileAndDiagnose(`
import "@typespec/openapi";
using OpenAPI;
@operationId("foo")
model Foo {}
`);

expectDiagnostics(diagnostics, {
code: "decorator-wrong-target",
message: "Cannot apply @operationId decorator to Foo since it is not assignable to Operation",
});
});