Working with Yarn PnP

[Downchuck]

Related to the question of why there is a tsp install in the first place -- I am trying to boot up with a clean yarn pnp workspace - which is a bit of a challenge with the current typespec ecosystem as it seems to lean into using "node_modules" heavily for extensions.

For the vs code extension, in settings.json file can be pointed to an unplugged compiler, e.g.:

  "typespec.tsp-server.path": "${workspaceFolder}/sdk/.yarn/unplugged/@typespec-compiler-npm-0.60.1-b04a9a8287/node_modules/@typespec/compiler"

Overall though, simply running yarn tsp does not seem enough to maintain the pnp resolver:

> yarn tsp compile .        
(node:39488) [MODULE_NOT_FOUND] Error: @typespec/compiler tried to access source-map-support, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.
(Use `node --trace-warnings ...` to show where the warning was created)
TypeSpec compiler v0.60.1

Diagnostics were reported during compilation:

/sdk/main.tsp:1:1 - error import-not-found: Couldn't resolve import "@typespec/http"
> 1 | import "@typespec/http";

Note source-map-support is in the dev dependencies, so that may be a hint as to something going strange in the loader.

Similarly, or from a different strategy, we can quickly reproduce a sad time with yarn in:

yarn dlx -p  @typespec/compiler -p @typespec/http tsp compile .

That stated, this appears to work:

yarn pnpify tsp compile .

That's using @yarnpkg/pnpify and whether or not it's needed, I added to my .yarnrc.yml:

packageExtensions:
  "@typespec/compiler@*":
    dependencies:
      "source-map-support": "~0.5.21"
      "@typespec/http": "*"
      "@typespec/openapi": "*"
      "@typespec/openapi3": "*"
      "@typespec/json-schema": "*"

[timotheeguerin]

It tries to load source-map-support if it is present(e.g. development) but ignore if not. I have never seen this warning, which version of node are you using?

For the yarn pnp part I am not sure, I am not familiar at all with it, how does it work? Could you make a repo with a repro so we can investigate how to support that.

[Downchuck]

Fell off on this thread - adding to it, hoping I can find a conclusion as I hack a bit -

This one ought work on start up, from yarn unplug @typespec/compiler:

  "typespec.tsp-server.path": "${workspaceFolder}/.yarn/unplugged/@typespec-compiler-npm-0.66.0-dd51960eb2/node_modules/@typespec/compiler"

But still seeing an issue:

2025-03-12 15:00:17.583 [info] Server path loaded from TypeSpec extension configuration: ${workspaceFolder}/.yarn/unplugged/@typespec-compiler-npm-0.66.0-dd51960eb2/node_modules/@typespec/compiler
2025-03-12 15:00:19.304 [error] TypeSpec client: couldn't create connection to server.
2025-03-12 15:00:19.305 [error] Launching server using command tsp failed. Error: spawn tsp ENOENT
2025-03-12 15:00:19.305 [error] TypeSpec server executable was not found: 'tsp' is not found. Make sure either:
 - TypeSpec is installed locally at the root of this workspace ("c:\...") or in a parent directory.
 - TypeSpec is installed globally with `npm install -g @typespec/compiler'.
 - TypeSpec server path is configured with https://typespec.io/docs/introduction/editor/vscode/#configure.
2025-03-12 15:00:19.305 [error] Error detail Launching server using command tsp failed. Error: spawn tsp ENOENT

This one is not as surprising, the VS Code Generate OpenAPI tool - needs to know to run yarn instead of npm/node via some configuration flag:

2025-03-12 15:03:32.139 [info] Generate from entrypoint file: ...\sdk\main.tsp
2025-03-12 15:03:36.698 [info] select @typespec/openapi3
2025-03-12 15:03:36.730 [error] '"node"' is not recognized as an internal or external command,
operable program or batch file.
2025-03-12 15:03:36.732 [error] Error getting dependencies. {"stdout":"","stderr":"'\"node\"' is not recognized as an internal or external command,\r\noperable program or batch file.\r\n","exitCode":1,"error":"npm view @typespec/openapi3 dependencies peerDependencies --json failed with exit code 1","spawnOptions":{"shell":true,"stdio":"pipe","windowsHide":true,"cwd":"c:/.../sdk"}}

[Downchuck]

For a simple yarn reproduction, this ought to work:

using TypeSpec.Http;

/main.tsp:1:1 - error import-not-found: Couldn't resolve import "@typespec/http"
> 1 | import "@typespec/http";

The typespec compiler one-way-or-another is escaping the module resolution process. I can try and track that down - as I see there is normal enough code for trying to use node module resolution - it's just missing it somewhere.

source-map-support is in devDependencies:

https://github.com/microsoft/typespec/blob/766e53df2e5669a439b22c790fb5fcd007a111ad/packages/compiler/package.json#L130

It's an optionalish runtime dependency - this code is a bit odd:

typespec/packages/compiler/src/core/cli/cli.ts

Line 4 in 766e53d

await import("source-map-support/register.js");

try {
  // eslint-disable-next-line @typescript-eslint/ban-ts-comment
  // @ts-ignore
  await import("source-map-support/register.js");
} catch {
  // package only present in dev.
}

By calling it out as a dependency, and using yarn pnpify to handle the more stubborn bypass on module resolution, things compile.

yarn pnpify tsp compile .

I suspect a tweak here and there would allow yarn run tsp and yarn dlx to work as expected. In both sets of code, I think the changes would be technically correct, as it seems it's a missing hook into the node resolver, and that conditional source-map-support hook for "dev" only does not seem to be doing any favors.

[timotheeguerin]

It's an optionalish runtime dependency - this code is a bit odd:

tbh I am not sure how else you can do that with ESM modules. We do rely on this in a few other places to provide optional peer dependencies, like using the xml library in openapi3 emitter.

regarding bypassing teh module resolution process, yes typespec has it own resolver that is needed because we need to resolve our own files as well. It follows the node module resolver algorithm as described on their website but I assume yarn pnp does some things to make their system work and to support it we'd need to also explicitly resolve it.

[Downchuck]

Here's a cross-link / note on custom yarn pnp resolvers - I don't think that's needed here, but I do think the code may show where there was a miss on calling the node resolver chain.

yarnpkg/berry#6604

Typespec is working absolutely wonderfully for us with AWS lambda functions. Absolute dream. Just a few work-arounds with yarn pnp due to custom resolvers in both that aren't quite hooking in.