New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Concerns with TypeScript 4.5's Node 12+ ESM Support #46452
Comments
I think this issue is a good example for the long needed plugin (hook) system. The Solution to the first 2 Problems is rollup at present you can use it with plugin typescript to resolve anything correct and then inject it into the typescript program. i am already researching how i could maintain and release a typescript-rollup version which would be typescript + rollup hooks and plugins. Conclusiona Plugin/Hook System is the Solution for the Resolve Problem. The Only one that is flexible and adjustable enough to cover every case. |
There is already a hooks system built into package.json... it's bad, but the whole point is to get rid of it as soon as dependencies merge the PR's to fix the issues. I have the following install hook as a bandaid while upstream applies the fixes to default imports and reachable types/modules.
I'd rather have 4.5 stable sooner, than to wait for yet another workaround. |
@rayfoss we can take your example to again show that a plugin/hook system like the one from rollup is badly neeeded. you mixed linux shell script into your package.json as workaround. |
Yeah, so the trouble is if TypeScript doesn't encourage the use of This is something I've mentioned in the past on some related issue, but has it been considered not to have a distinct By having both |
This seems like the absolute worst conclusion to ever reach. I'm really sorry for butting in on this issue as not a huge avid typescript users but the one thing I like about typescript is precisely it doesn't require 9001 packages like soo many other builders and bundlers to actually get into a working state. Typescript is standalone that just-works and doesn't require the end programmer to have to build-their-own-compiler themselves. Add plugins is just gonna be that, making it more and more complicated while adding nothing really. Especially something like this issue that REALLY needs to work out of the box but isn't. And saying to people "oh you're using typescript but typescript is dumb like bundlers, you need to hook plugins to get it working" is something you don't want to say to developers or people. I really strongly advice against any case of adding plugins to this package. If it doesn't work, we can fix it. If it works, why would you need a plugin just to make configging even more complicated? Best regards: P.S. |
@TheThing in my case it is simply needed because there are many package authors with total diffrent opinions and i do not want to get blocked by them. I also do not want to hardfork everything and so on. The only Alternativ to a plugin system in typescript is the usage of dev bundels that are typescript compatible. also my conclusion is driven from the fact that there are tons of other environments not only nodejs i only vote for resolve hooks and plugins because of all the diffrent environments as also package managers. |
What about removing Node12 and starts from Node 14? |
One note here from https://github.com/microsoft/TypeScript/issues/46550#issuecomment-954348769—for users who have declaration emit enabled, error messages like this are probably going to be a common symptom of a dependency that needs to update their export map to include
It feels pretty non-obvious that that’s what’s going on so I wanted to make a note of it here. Basically, the declaration emitter wants to print a type like |
|
Oh. Yes, I found that package actually exports the correct typing and JS file at |
There are some rough edges with the new resolution mode. I've been trying out this mode for two weeks. The main issues I've hit are:
Despite all of these issues, I still hope that the new resolution algorithm becomes available on stable TypeScript: the core seems good and bugs/UX can be iterated. The new resolution mode lets me get rid of tons hacks and makes it finally ergonomic to use ESM natively.
I was able to use In my personal experience, all of these improvements strongly outweight the current issues: it makes things so much simpler as it realigns TS behavior's with Node's. I hope that the current issues get fixed soon and I hope that the concerns will not delay the new resolution too much. |
I wanted to link the issue I've double checked my message: I linked it as |
To align with `tsc` default behavior, mentioned in microsoft/TypeScript#46452 Fixes vitejs#3040
What if TypeScript 5 assumes It would be regrettable if we have to add |
Just to be clear it is Node (not TypeScript) that requires |
at present i think the type fild in the package.json is less relevant as that is only a switch for the .js extension inside NodeJs Typescript at present 4.5+ detects the module type via Typescript is not a NodeJS only Product at last i guess that. ps i still have Javascript Projects without a package.json at all and i use the global installed typescript to typecheck them it works great and it should stay working. |
For folks who are interested in testing esm-node out: npm add typescript@4.5.0-dev.20211101--save-dev
yarn add typescript@4.5.0-dev.20211101 --dev
pnpm add typescript@4.5.0-dev.20211101 --dev This is the closest nightly npm release to the RC, so can act as "4.5 but with ESM enabled" for your projects. |
@fbartho I don’t quite understand your use case. Why is it that you want to migrate from CJS to ESM file-by-file? Are you writing a self-contained app or a library that others will consume? If the latter, are you trying to provide entrypoints in CJS, ESM, or both?
This is definitely a pain, but to be clear, it has nothing to do with TypeScript. There is no way any kind of compatibility mode TypeScript could dream up can get you out of the fundamental async pickle of needing ESM-only dependencies in a CJS context.
No. That’s literally asking to convert an async operation to a synchronous one. This is the interop model that Node chose, which @weswigham (among others) advocated strongly against. But at this point it’s 100% out of our hands. All we can do now is relfect the reality that Node gave us. |
Maybe I'm asking the wrong question. We have a small dozen of nodejs backend applications (cloud functions, express servers, local tools, graphql servers, and frontend react sites), and another small dozen of library packages in a monorepo. We depend on libraries from npmjs some of which are going ESM-only. We don't give a shit about CJS or ESM. But we do feel responsible for keeping our dependencies up to date, and weekly it seems like we have to mark more and more dependencies as "can't update this one anymore". What's the right move here? How do we go from "everything is CJS" to whatever future allows our dependencies to stay up to date (via dependabot or similar)? |
Gotcha. So, just in case it wasn’t clear, ESM is importable from CJS, but it must be done with dynamic imports, which return a Promise. So, if all of your stuff is already async, you may be able to get away with Migrating some, but not all, of your files to ESM will not help you with this, because as long as you have any CJS that depends on anything ESM, you’re going to have the infective async problem. Migrating all of your code to ESM will theoretically get you out of trouble, so I suppose you’re wanting to migrate piecemeal just because migrating all in one go is impractical. It is true that TypeScript is currently going to make it difficult for you to migrate TSX files to ESM one at a time because we lack format-specific extensions that support JSX syntax at the moment. Like @cspotcode said, the only way you can control the module format of TSX files is with the That said, if you are set on eventually migrating everything to ESM, it might not be the worst to do big batches (whole sites/component libraries) of TSX files at one time, since these components surely depend on each other a lot—migrating them file by file might actually be more work than doing them all together. |
You say "if" like I have a choice, but I don't really think I do, dependencies are updating in breaking ways, and I can't exclude them forever, and I don't control the ecosystem packages that rely on them. ¯_(ツ)_/¯ So let's say hypothetically, I'm willing to do this in batches. (And let's ignore Everything shares a top-level tsconfig. Each application/library has its own package.json.
|
This is a conscious choice by some package maintainers to try to use their package's popularity to lurch the ecosystem forwards, regardless of how much it harms their users. The node maintainers assumed package authors wouldn't be user hostile and would be willing to ship so called "dual mode" packages that had both esm and cjs sources for a time (or that most users still using cjs would be fine with old package versions), and that those would bridge the gap. In reality, there's little incentive for a library author to do so beyond perceived "legacy user" demand. |
For packages that are going to run in Node, setting Assuming that all your code is currently compiling to CJS, you shouldn’t make any package.json changes—this will keep For your frontend apps and libraries, you should really consult with whatever bundler you’re using. It may be compiling your dependencies’ ESM code down to something synchronously available by merit of being bundled. These new modes in TypeScript are made specifically for Node. When bundlers do magic stuff, all bets are off.
In ESM files, yes. There are no directory-index imports in ESM. Let me stress again—while I’m happy to give you advice, even in this thread here, beyond setting |
Clarification: this implies correspondingly named values for |
I cannot even get ts-socery.mov |
Are you using bundled TypeScript from vxc (It could be outdated)? Make sure to use your workspace tsdk with this settings: {
"typescript.tsdk": "node_modules/typescript/lib",
} Place it in |
Even compiling with |
What is the recommended way to upgrade large codebases to ESM? I haven't found anything in the docs. The most painful part is the no extensions imports to By the way there should be a much bigger and clearer warning that you should use |
There isn't one. "Upgrading" to esm is a breaking change for a node package, no way around it. |
Yes, but a codemod or similar could exist to add the Both of these things are completely feasible and I'm asking if there's already something like that. There's no reason to manually add |
I mean, there are a bunch of community codemod tools that do stuff like that. We don't exactly bundle tools to make moving from target to target easier; it's just not a thing we do. |
I understand not bundling codemods, but I was wondering if there was an official or unofficial recommendation on how to migrate to ESM. Now that some popular packages are moving to ESM only, you can probably expect more and more people going from CJS to ESM. Expanding the docs with a page featuring a guideline or recommendations for migrating to ESM would be very helpful, it will avoid many future Github Issues. Currently this page covers the technical basics regarding TS and ESM, but a page full on dedicated to migrating would be helpful to many people in the future. Such page could mention the existence of tools that aid migration, or TS settings that help. From what I've seen the TS team has decided against any setting that allows After all the vast majority TS codebases currently use CJS, and in the future if they want to keep up with the npm ecosystem they'll presumably move to ESM. So good docs on this would help numerous projects. |
There is a setting, and it’s called “using CommonJS” It may sound like we’re being pedantic here, but our point of view is derived from two principles which may not be obvious to many people reading up on the new
This is core to why we don’t have a migration guide. If folks are set on migrating to ESM and seeking guidance, I would encourage them to look for general JS/Node guides on the subject—it shouldn’t have to be specific to TypeScript. If you hit pain-points that are TypeScript specific (reminder: this does not include adding extensions to module specifiers; this is literally a Node requirement), we’re interested in hearing those—they may be good candidates for docs (if not actual fixes in our code). (FWIW, the area I’m most concerned with here is dealing with dependencies—it’s going to take some time before all the widely used npm packages who use conditional exports get their typings properly configured. For a user migrating to ESM who runs into dependency issues of this nature, there’s not much they can do beyond recognize what the problem is and file an issue with the library, but maybe we can help with that diagnostic step somehow.) |
I don't think anybody with a large codebase wants to migrate for the sake of migration. TS already gives you But I understand TS is against that and I don't want to start another flamewar with something that has been discussed many times. So given that some people will need to migrate to ESM, and such people will need to change their imports, I think there should be docs regarding this. And these docs could rephrase what you just explained, that you must change your imports so that they are imports that make Node happy. But it is a typescript specific step because people are used to TS handling their imports. Sure, once you know how TS with ESM works you can say imports are now not a step specific to TS. But where are you going to get that knowledge? People looking at general JS guides won't know how TS decided to handle ESM. They can at most make assumptions. And a lot of people, if they assume that now you have to use extensions with I assumed that and many people will assume that too. I'm not arguing against that decision, just stating that there should be good documentation so people don't make these wrong assumptions. What I propose is having some documentation with something like a checklist for migrating, something like
I see no downside to having such documentation, and while you can argue that you can infer all that from here, having this info concisely presented to those wanting to migrate will help a lot of people, and we'll avoid a lot of erroneous github issues. |
In NodeJS ESM is only a good Authoring format not a good Runtime format at all. The Pros of ESM while coding
The parsing and loading speeds of ESM and CJS do total depend on the overall project variables like size module graph time of instantiation. Cons ESM:
Update because thumbs downI did not design that module system that was ECMA i only say it . Thats how it is. i do not say that ESM Got well designed. |
I don't really get it, you've made a new major version that doesn't support the most trending JS language, TypeScript? |
Here is a single-file script that automatically adds https://gist.github.com/Jack-Works/6ebba47c3fdef50637b10819f3e7e68a |
I just want to give a little update:
this introduces the next problem: |
Hey @andrewbranch what are the remaining concerns keeping this issue open? Most sub tasks seem closed now but not sure if I’m missing something. |
For TypeScript 4.5, we've added a new module mode called
node12
to better-support running ECMAScript modules in Node.js. Conceptually, the feature is simple - for whatever Node.js does, either match it or overlay something TypeScript-specific that mirrors thes same functionality. This is what TypeScript did for our initial Node.js support (i.e.--moduleResolution node
and--module commonjs
); however, the feature is much more expansive, and over the past few weeks, a few of us have grown a bit concerned about the complexity.I recently put together a list of user scenarios and possibly useful scripts for a few people on the team to run through, and we found a few sources of concerns.
Bugs
Most complex software ships with a few bugs. Obviously, we want to avoid them, but the more complex a feature is, the harder it is to cover all the use-cases. As we get closer to our RC date, do we feel confident that what we're shipping has as few blocking bugs as possible?
I would like to say we're close, but the truth is I have no idea. It feels like we'll have to keep trying the features for a bit until we don't run into anything - but we have less than 3 weeks before the RC ships.
Here's a few surprising bugs that need to get fixed before I would feel comfortable shipping
node12
in stable.resolveJsonModule
can't be used withnode12
: Typescript 4.5 resolveJsonModule should also work with node12 and nodenext module and resolveTypes #46362.ts
and.tsx
files innode12
(unfiled, reported by @andrewbranch)package.json
changes in packages not tracked (unfiled, reported by @DanielRosenwasser)UX Concerns
In addition to bugs we found, there are just several UX concerns. Package authoring is already a source of confusion in the TypeScript ecosystem. It's too easy to accidentally shoot yourself in the foot as a package author, and it's too hard to correctly consume misconfigured packages. The
node12
mode makes this a whole lot worse. Two filed examples of user confusion:"moduleResolution": "node12"
of Typescript 4.5 does not work as expected #46408export
field is confusing to configure and diagnose: Surprising (or incorrect) priorities with package.json exports fields? #46334While there might be a lot of "working as intended" behavior here, the question is not about whether it works, but how it works - how do we tell users when something went wrong. I think the current implementation leaves a lot of room for some polish.
But there are some questions about this behavior, and we've had several questions about whether we can simplify it. One motivating question I have is:
When a user creates a new TypeScript project with this mode, when would they not want
"type": "module"
? Why? Should that be required by default?We've discussed this a bit, and it seems a bit strange that because we want to cover the "mixed mode" case so much, every Node 12+ user will have to avoid this foot-gun.
I would like to see a world where we say "under this mode,
.ts
files must be covered by a"type": "module"
"..cts
can do their own CommonJS thing, but they need to be in a.cts
file.Another motivating question is:
Why would I use
node12
today instead ofnodenext
?Node 14.8 added top-level
await
, but Node 12 doesn't have it. I think this omission is enough of a wart that starting at Node 12 is the wrong move.Ecosystem
The ecosystem is CONFUSING here. Here's a taste of what we've found:
.js
extensions, but TypeScript expects them. Not all of these can be configured with a plugin..ts
extensions, but TypeScript doesn't allow them!export
fields, but don't have atypes
sub-field withinexport
(e.g. RxJS, Vue 3).export
fields, but their@types
package might not reflect that.The last two can be easily fixed over time, though it would be nice to have the team pitch in and help prepare a bit here, especially because it's something that affects our tooling for JavaScript users as well (see #46339)
However, the first two are real issues with no obvious solutions that fall within our scope.
There's also other considerations like "what about import maps?" Does TypeScript ever see itself leveraging those in some way, and will package managers ever support generating them?
Guidance
With
--moduleResolution node
, it became clear over time that everyone should use this mode. It made sense for Node.js apps/scripts, and it made sense for front-end apps that were going to go through a bundler. Even apps that didn't actually load fromnode_modules
could take advantage of@types
in a fairly straightforward way.Now we have an ecosystem mismatch between Node.js and bundlers. No bundler is compatible with this new TypeScript mode (and keep in mind, back-end code also occasionally uses a bundler).
Here's some questions I wouldn't know how to answer confidently:
node12
ornodenext
?"type": "module"
- should we always recommend.mts
?Next Steps
I see us having 3 options on the table:
The text was updated successfully, but these errors were encountered: