Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support `.mjs` output #18442

Open
demurgos opened this Issue Sep 13, 2017 · 78 comments

Comments

Projects
None yet
@demurgos
Copy link

demurgos commented Sep 13, 2017

Experimental support for ES modules just landed in Node 8.5 (changelog, PR).
Since ES modules have some parsing and semantic differences, Node decided to use the mjs extension for ES modules (while js is for the "script" target and commonjs modules).

The current Typescript version (2.5.2) supports ES modules emission but uses the js extension by default. It means that to use it with Node, a post-compilation step is required to change the extension from js to mjs. This adds undesirable complexity to use native ES modules support with Node.

A solution would be to add a compiler option to output *.mjs files when emitting ES modules.

Edit (2018-03-22): The propositions below are a bit outdated. I recommend reading the issue to see the progression. See this comment for my current proposition.

Notes:

  • It is possible that new parse goals will be added in the next versions of the ES spec. Letting the user provide the extension explicitly (as opposed to a boolean switch) would allow to be forward compatible with other new extension.
  • It should still be possible to emit ES modules with the *.js extension, many tools rely on the js extension.
  • Another solution would be to have a different extension for the source files: *.mts files would compile to *.mjs, this would be similar to *.tsx and *.jsx.
@kitsonk

This comment has been minimized.

Copy link
Contributor

kitsonk commented Sep 13, 2017

See #10939 (and #9839, #9551, #7926, #7699 and #9670) and this comment.

@demurgos

This comment has been minimized.

Copy link
Author

demurgos commented Sep 13, 2017

@kitsonk
Thanks for posting these link but I am not sure if they are relevant, could you explain them? I found these while checking if the issue was already opened: most of the issues you listed are about input files. This issue is about compiling ts files to mjs instead of js (adding an options to change the compiler output).

@kitsonk

This comment has been minimized.

Copy link
Contributor

kitsonk commented Sep 14, 2017

Basically those say that TypeScript does not want to concern itself with managing extensions like that, as it is already complicated enough. In particular the output extensions run afoul of the TypeScript non-goal of:

  1. Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.

If you want .mjs files, it would be best to do something like:

$ npm install renamer -g
$ renamer -regex --find '\.js^' --replace '.mjs' './outDir/**/*.js'
@demurgos

This comment has been minimized.

Copy link
Author

demurgos commented Sep 15, 2017

Thank you very much for clarifying your comment.
I understand that adding any new feature is a burden since it means that it then must be supported for a long time, but I believe that the benefits of the mjs support are worth it: it won't turn tsc in a complex build tool but help TS users.

The long term goal of the Node team is to allow users to author their code using ES modules without paying a cost due to the commonJs modules and script target. This is one of the reasons why the proposals to require "use module" or an export statement (even an empty one) were dropped in favour of using a new extension. See the Node EPS discussions in nodejs/node-eps#57 and nodejs/node-eps#60. If a post-compilation step to rename the files is still required in two years only because .mjs was introduced later then it will just add another burden to remember for years (like the BOM or the different line endings)... Even if there are some discussions to support ES modules with the .js extension ("module" property), the current position is that - if such support is added - it should be used as a fallback for old tooling. .mjs is promoted for new code.

Complex build tools have their place for assets management, dead-code elimination, bundling, minification, etc. These all fall out of scope for TS, but being able to produce runnable code does not. Big projects have their own workflows and use the TS library (either directly or through plugins for other task runners), adding the renaming step is not a big deal for them. On the other hand, a sizeable chunk of projects using Typescript are small/medium sized libraries. These libraries usually only need a single tool: tsc. If they want to support native ES modules, they'll currently have to come up with a command similar to the one posted above by @kitsonk. The problems are that it raises the barrier to entry, causes duplicate effort and makes the build more expensive.

It raises the barrier to entry because newcomers can no longer simply use tsc -p && node index.mjs (--experimental-modules will no longer be required when the use-case for this issue will be relevant) and run their code to try out Typescript. For existing projects, the "build and run" command is usually already defined as an npm script or gulp/grunt/webpack/whatever task, writing this command (or understanding it) will be more difficult: for some persons it will be trivial, for others it'll require a few hours of research. The rate of change in the JS ecosystem is already pretty high, let's try to not worsen it.
Now, a related problem is that it may cause some inconsistencies and duplication: different projects will use different ways to rename their file. I discovered renamer thanks to the message above, but I would have written a gulp task otherwise. Someone else would have used a POSIX-only shell command with broken edge cases or rolled their own helper Node script. To be honest, I think that most good projects would have a good implementation but this lack of standard way to deal with file renames when no build system is already there may increase the "barrier to entry" problem.
Finally, once the manual renaming is configured, it still represents an additional cost. I don't have any measures but for medium projects and using Node to rename the files, it may be longer to start the VM than running the file rename: why not doing it in the same process as TS? Also: what about --watch? How do I make it play well with my custom command? Do you have to have to watch the build directory to rename as the files are emitted?

So far, most of my arguments were about the build complexity: having it done once in TS is better than having each project coming up with its own solution. There is another important reason to have ES modules working out of the box, and I think that it aligns with the goals of TS:

  1. Align with current and future ECMAScript proposals.
  2. Preserve runtime behavior of all JavaScript code.

ES modules are part of the spec and the TS syntax uses them. These modules have a specified runtime behaviour that cannot be fully replicated in commonJs. It affects among other things circular dependencies, early errors, mutation of exported namespace, etc. Many people won't care, until it bites them. When setting the module option to commonjs, TS does a pretty good job of generating an output with the expected behaviour but it does not trump using real ES modules. So we are in a situation where we can either set an option and have a good approximation using commonJs but if we want the exact spec and runtime behavior we have to jump through hoops and loops.

One last argument is that even if browsers support native ES modules without requiring .mjs, real-world usage of native ES modules will start server-side with Node because you can control its version. Node is important for the whole ecosystem so it would be damageable to just ignore that it uses .mjs for ES modules. You can't just treat it as a "platform detail" when this platform is used by the majority of the TS audience.

To summarize, here are my main points:

  • Now that native support exists for ES modules, people will want to use it out-of-the-box.
  • Node settled on .mjs and hope that it'll gain traction. .mjs is Node-specific, but Node is important
  • Many libraries only need tsc, requiring a second tool for renaming increases barrier to entry and causes duplicated effort (of varying quality).
  • Using a post-compilation command leads to higher build times and does not play well with the watch mode.
  • Using native ES modules is the only way to match the spec exactly, TS should not make it hard to use
@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Sep 28, 2017

Just an Opinion

As a very strong fan of TypeScript, and obviously a user of node (I guess even TypeScript is) I would really like to see some agreement between them on an issue that has had so much disagreements over many years of rational and sometimes awkward discussions.

I would have loved it if node had a transition phase to make common js (the non-standard) become .cjs over a period of two years.

That said, mjs seems to be the current direction, obviously that non-js file extension (either one) itself does not matter, but the ability for TypeScript to provide a way to control this inline as the originator of the transpiled files is something that really falls to TypeScript.

That said (again), mjs as a locked extension is kind of very opinionated, someone must have realized that forcing js was okay, it was already called js, but forcing mjs, come on, even my spelling checker is nagging me on that one. I hope nodejs will become a little more flexible on that one.

Proposal

Please make it possible to specify mjs as a module format that simply means compiling es2015 modules and calling them mjs :)

@trxcllnt

This comment has been minimized.

Copy link

trxcllnt commented Oct 3, 2017

It's important to note here that .mjs isn't only about file extensions. running node with the
--experimental-modules flag also means that ES6 module specifiers must be valid urls, so the extension has to be included in the import statement, e.g. import x from './x.mjs

If I'm understanding this right, renaming the files isn't a solution, TS would also need to compile module specifiers with the .mjs extension in the ES2015+/mjs mode. We're keen to support newer versions of node by publishing non-downleveled iterators as es2015+ modules for IxJS, so it would be great to get an answer on this soon.

@dpogue

This comment has been minimized.

Copy link

dpogue commented Oct 3, 2017

TypeScript already allows you to include a .js extension on your imports (even when you're actually importing .ts TypeScript files). The change needed would be to expand that to also allow .mjs.

@trxcllnt

This comment has been minimized.

Copy link

trxcllnt commented Oct 3, 2017

It's not about the source files including the extension, it's about the compiled files.

If I compile import x from './x' into ES5/CommonJS, TS emits:

var x = require("./x");

If I compile it to ES2015/ESModules with an .mjs extension, TS should emit:

import x from "./x.mjs"

If a library is trying to support both old and new node with CommonJS and ESM side-by-side (as *.js and *.mjs extensions respectively), node only imports the .mjs files if the extension is included in the module specifier.

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Oct 3, 2017

There are some potential updates in the works… including a --loader option that would you to specify a file with a resolve hook that will take the specifier and parent module path, then it could handle extension prioritization as needed. It seems to be very flexible but their goal is to make it declarative and not have people overloading the actual loader functions.

In terms of picking mjs over js, that is only the case in the CJS loader system which uses the now legacy Module prototype with it's hooks, the ones that everyone likes override all the time.

In the current --experimental-modules release, the ESM loader is hard coded to '.mjs' or falls back to the CJS loader.

What I've seen so far over the past few days makes me believe that the ESM loading system will take over with pluggable CJS loading (unless opting to use the legacy-modules by flag or based on the main file).

Check out this PR/discussion

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Oct 3, 2017

@trxcllnt I guess regarding the part about baking the extension right into the import… this applies to import/export everywhere now as part of loader specs.

When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.

So if TypeScript would continue to work without third-party tooling, it will need to find a strategy to write out standard (not ISO but platform-specific) out-of-the-"compiler"-box projects that will just work in either node or browsers at least depending on the compilerOptions intent specified by the user, and do so without asking the user to mockup some hack to get it to work.

This is just an opinion, but honestly, it feels like it for TypeScript to address.

Checkout this MDN reference but make sure you notice the part where it first said "excluding the .js extension" then in the examples included the extension anyway.

@bmeck

This comment has been minimized.

Copy link

bmeck commented Oct 3, 2017

I'd like to include here that the IANA has an Internet Draft which specifically adds .mjs for COMMON usage that represents the Module goal of ECMAScript . .mjs is not purely a Node.js concern and is even included in an example for browser specs and is supported by a variety MIME DBs already like shared-mime-info.

@daflair can you expand on what

(unless opting to use the legacy-modules by flag or based on the main file).

means?

The next step after that PR is to setup per package loaders to be able to guard against global loader mutation. So, it might be going the direction you think.

When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.

Interestingly the WHATWG Loader Spec had a very early version in the ECMAScript spec that was removed at the last minute before ES2015! There are interesting other things like the original CommonJS Spec which mandated not to have extensions. We should probably avoid dwelling on the past so much since the different loaders vary so much on these opinions.

Node's EP specced a superset of the WHATWG resolve algorithm that does do various Node idioms like file extension completion. However, the browser has a subset that is safe to use in Node. Still, even with that subset there are problems with dependency trees since "bare" imports are waiting on userland feedback (you can get involved in the hook here), but is mostly left up to intelligent servers and service workers for now.

I'd recommend trying to compile down to the WHATWG compatible specifiers except for bare specifiers for now.

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Oct 3, 2017

@bmeck what I meant by:

unless opting to use the legacy-modules by flag or based on the main file

I was making an assumption that at some point you will have to use something like --disable-es-modules or --legacy-modules which is really an option that simply does the opposite of --experimental-modules (when this flag is the default behaviour) to resort the current loader system and completely bypass anything related to the new loader (ie legacy applications that simply find ways to be incompatible with this existential change to their eco system).

But now that I dug in a little deeper in your recent PR's this might not be your intent…

So it is best to ask you about your intent here 😉?

@bmeck

This comment has been minimized.

Copy link

bmeck commented Oct 3, 2017

@daflair I'm still not understanding. ESM support in --experimental-modules completely avoids touching CJS except by overtaking the defaulting of .mjs to CJS (it now throws). I can't think of a reason to introduce such a flag. Also, it would be a very hard sell to change the default behavior from CJS to ESM or vice versa and I doubt that will ever happen.

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Oct 3, 2017

Okey, when we get to the point where --experimental-modules is no longer needed, at that point, will the CJS loading system that exists today still remain (mostly) unchanged?

If so, at that point, without the above flag and without any other flags like --loader, …etc :

a. Would running with a .mjs entry do what it does today with the flag and the CJS loader will simply pass it to the ESM loader? meaning that all dependencies will be marshalled by the ESM loader (even if it delegates something to the CJS loader).

b. Would running with a .js entry always imply CJS (again no special flags) and this also means that ESM loader is essentially completely idle for the lifespan of the process (assuming there will be no require() support for es-modules in the future).

@bmeck

This comment has been minimized.

Copy link

bmeck commented Oct 3, 2017

Okey, when we get to the point where --experimental-modules is no longer needed, at that point, will the CJS loading system that exists today still remain (mostly) unchanged?

Yes, it is unchanged except the .mjs reservation.

a. Would running with a .mjs entry do what it does today with the flag and the CJS loader will simply pass it to the ESM loader? meaning that all dependencies will be marshalled by the ESM loader (even if it delegates something to the CJS loader).

No, the 2 loaders are decoupled except when ESM defers to CJS. Loading via import will always go through the --loader hooks. Loading via require will not go through the --loader hooks. Use import for green code going forward.

b. Would running with a .js entry always imply CJS (again no special flags) and this also means that ESM loader is essentially completely idle for the lifespan of the process (assuming there will be no require() support for es-modules in the future).

Yes .js is CJS. However, you can use import() to get a hold of ESM since that is available in all JS (even eval).

@kitsonk

This comment has been minimized.

Copy link
Contributor

kitsonk commented Oct 3, 2017

And people wonder why the TypeScript team wants to avoid getting into this area at the moment... 🙄 No it is, yes it isn't, no it is Even if TypeScript gets further into an extension mangling business it would wait until it was clear that --experimental would be retired as a flag and established as a default with appropriate battle hardened semantics.

@bmeck

This comment has been minimized.

Copy link

bmeck commented Oct 4, 2017

@kitsonk

🙄 No it is, yes it isn't, no it is

can you clarify what is unclear in the messaging from Node?

@trxcllnt

This comment has been minimized.

Copy link

trxcllnt commented Oct 4, 2017

@bmeck so one thing that seems like a major foot-gun is how node doesn't automatically select the mjs files when importing a module with --experimental-modules mode on. I understand the story on importing ESModules from the node_modules folder isn't complete yet (should it look for the mjs file path from "module" in package.json? who knows). But automatically selecting the CommonJS form of a module then converting its module.exports object to the default export means that code that works fine in other ESM environments is broken in node:

// node_modules/ix/Ix.mjs (no default export)
export class Iterable {};
// node_modules/ix/Ix.js:
module.exports = function Iterable() {};
// run with node --experimental-modules some_file.mjs
// works fine
import { default as Ix, Iterable } from 'ix/Ix.mjs';
assert(typeof Ix == 'undefined') // true, great
assert(typeof Iterable == 'function') // true, great
// broken
import { default as Ix, Iterable } from 'ix';
assert(typeof Ix == 'undefined') // false?
assert(typeof Iterable == 'function') // false :(
@bmeck

This comment has been minimized.

Copy link

bmeck commented Oct 4, 2017

@trxcllnt it is complete and has been laid out in depth in https://github.com/nodejs/node-eps/blob/master/002-es-modules.md

should it look for the mjs file path from "module" in package.json?

no.

But automatically selecting the CommonJS form of a module then converting its module.exports object to the default export means that code code that works fine in other ESM environments is broken in node

Correct, because this code cannot be spec compliant. There is absolutely no way to link it ahead of time since you need to eval to get the shape of module.exports. Emulations using CJS semantics like babel or typescript are not enforcing the shape of a module during Instantiate.

This style of eval based shape detection will never be possible.

In the future there may be a pragma that allows parse time instead of eval time declaration of shape, but that would still be a breaking change.

Simply put:

works fine in other ESM environments is broken in node

Is only true because those environments are not valid ESM

@trxcllnt

This comment has been minimized.

Copy link

trxcllnt commented Oct 4, 2017

@bmeck I'm afraid I wasn't clear. I'm not saying node shouldn't use the exports from a CommonJS module as the default when --experimental-modules is on.

I'm saying that when I run with --experimental-modules and I import a module from node_modules that has both a CommonJS and ESModules form side-by-side, I would expect node to select the ESModule over the CommonJS one.

I have this expectation because I'm explicitly running node in ESModules mode, and using an ESModule import statement to import a package that exports an ESModule. But instead because node selects the CommonJS version (and then exports becomes default export), the code doesn't do what I expected it to do.

And to clarify why this matters, we're compiling es2015+ features (generators, async generators) down to ES5/CommonJS as the .js files, and compiling the es2015/ESModules as the .mjs files. The ES5 files have all the iterator downleveling codegen necessary for ES5, but ideally we can run node with --experimental-modules and use the native-generator version of the lib automatically.

@bmeck

This comment has been minimized.

Copy link

bmeck commented Oct 4, 2017

@trxcllnt it does search for .mjs first. https://github.com/nodejs/node/blob/e00a488731ed333c3cd0952acdfe85729b734fa5/src/module_wrap.cc#L37

Your specifiers are different in meaning and you can see if you use import from 'ix/Ix'; that .mjs is preferred. I assume your package.json is explicitly pointing to the .js but was unsure given the example not providing one.

@trxcllnt

This comment has been minimized.

Copy link

trxcllnt commented Oct 4, 2017

@bmeck npm install ix to check it out for yourself, but the relevant bits of the package.json are this:

{
  "name": "ix",
  "main": "Ix.js",
  "module": "Ix.mjs"
}

We can't set "main": "Ix.mjs", as that'll throw when not using --experimental-modules. Right now we set "module" to the mjs file because most of the major bundlers (webpack, rollup, etc.) will use the "module" path over "main". But from everything I've read so far, there isn't an ESM analog to CJS's "main" entry in package.json (please correct me if I'm under-informed on this).

@trxcllnt

This comment has been minimized.

Copy link

trxcllnt commented Oct 4, 2017

@bmeck and also I want to clarify, you and I are on the same side of this issue. .mjs is just fine with me, as long as things have reasonably normal default behavior. I'm in this thread arguing that TS should generate imports with .mjs extensions, because it seems all ESModule import specifiers are now required per the spec to be valid URLs.

Considering the prevalence of importing libraries from node_modules in node, I was a bit surprised there wasn't clarity on this issue for ESModules. And also that it also did a bad/unexpected thing when we're explicitly trying to support both CJS and ESM in the IxJS project.

I would even have preferred node to throw/print some sort of "ambiguous import" warning like "hey dummy, you tried to get an ESModule from a package.json file which doesn't have a way of specifying ESModule mains, here's the ES5 version but maybe be more explicit next time"

@bmeck

This comment has been minimized.

Copy link

bmeck commented Oct 4, 2017

@trxcllnt

This comment has been minimized.

Copy link

trxcllnt commented Oct 4, 2017

@bmeck HMMM yeah that sounds like a solution. sorry for hijacking the thread everybody.

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Oct 4, 2017

Awesome @trxcllnt and @bmeck, I guess we are all really trying to figure things out, so a resolution is always a great addition to a thread.

By the way, many of us nagging at Node to support ESM with .js are also nagging at TypeScript to support "module": "mjs" that can run with node directly without a --loader. Different workflows, different requirements.

So here is a silver lining:

// hybrid package.json
{
  
  "main": "index",
  "files": [
    "index.js",
    "index.mjs",
    "index.d.ts"
    "index.es.or.whatever"
  ],
  
}

Then if I do my work right, I would really appreciate being able to add this:

// hybrid package.json
{
  
  "scripts": {
    "compile-cjs": "tsc -m commonjs",
    "compile-mjs": "tsc -m mjs",
    "compile-whatever": "boink plug and play box yes no okay --but-follow-es-standards-please",
    "compile-all": "npm-run-all compile-cjs compile-mjs …"
  },
  
}

Instead of having to do this:

"compile-mjs": "tsc; rename …; find-replace … from /.*?/ with $1.mjs --verbose"

Does that sound about right?

@bmeck so those index variants will resolve to mjs if imported and js (cjs) if required when things are stable without any special flags, right?
@demurgos is this what you had in mind?
@trxcllnt is that in line with your conclusions so far?
@kitsonk are we any closer to convincing you?

👍 Let's keep at it!

@saschanaz

This comment has been minimized.

Copy link
Contributor

saschanaz commented Apr 24, 2018

Node.js ES module support is still experimental and requires a flag. Supporting it is not simple, even import { readFile } from "fs" won't work because Node.js gathers everything into default for CJS modules. I think using esm is the best before it gets stable enough to earn a good support from other tools like TypeScript.

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Apr 25, 2018

@saschanaz no question, you are absolutely right for use cases where using esm appropriate. The reality of this option is that it is compiling your ESM module to CommonJS. This works great for most mainstream use, but excludes being able to specifically test experimental TypeScript scenarios with real modules, which is the who purpose an experimental stage (where users can actually be contributing in places where developers might not have been looking).

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Apr 25, 2018

Since my last post (timestamped 4 hours ago now) I cloned TypeScript and did my best to take my experiment from hacking the compiled tsserver.js into the actual source.

This required very few changes than anticipated: (some changes are simply due to whitespace reformat/reflow)

 src/compiler/core.ts                  | 12 ++++++++----
 src/compiler/emitter.ts               |  3 ++-
 src/compiler/moduleNameResolver.ts    |  6 +++---
 src/compiler/program.ts               |  1 +
 src/compiler/resolutionCache.ts       |  2 +-
 src/compiler/types.ts                 |  1 +
 src/services/codefixes/importFixes.ts |  2 +-
 7 files changed, 17 insertions(+), 10 deletions(-)

Then for tests:

 src/harness/harness.ts                         | 18 ++++++++++++++----
 src/harness/projectsRunner.ts                  |  6 +++++-
 src/harness/unittests/reuseProgramStructure.ts |  8 ++++++++
 src/harness/unittests/tsserverProjectSystem.ts |  1 +
 4 files changed, 28 insertions(+), 5 deletions(-)

And testing: (needed a jake baseline-accept after eliminating other errors)

Discovered 103 unittest suites.
Discovering runner-based tests...
Discovered 12150 test files in 347ms.
Starting to run tests using 8 threads...
Batching initial test lists...
Batched into 8 groups with approximate total time of 2m6s in each group. (90.0% of total tests batched)

  [▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬] ✓ 60367 passing (2m31s)


  60367 passing (3m)

Linting: node node_modules/tslint/bin/tslint --project scripts/tslint/tsconfig.json --formatters-dir ./built/local/tslint/formatters --format autolinkableStylish
rm -rf tests/baselines/local/projectOutput/
Linting: node node_modules/tslint/bin/tslint --project src/tsconfig-base.json --formatters-dir ./built/local/tslint/formatters --format autolinkableStylish
✨  Done in 206.69s.

I will need to do some due diligence in the morning then open a PR so others with similar goals can collaborate on this.

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Apr 25, 2018

The PR passed all tests. Since it is only a PR, the actual built files are compiled into built/local. If you are interested in using it, I am putting together a wrapper package and working on a README here:

https://gist.github.com/smotaal/4f9458cf43cd7a55a12d5c476c15400e

@que-etc

This comment has been minimized.

Copy link

que-etc commented Apr 26, 2018

@SMotaal
Have you considered implementing a node loader hook, which would natively load .js files as ES modules?

Here is a working draft:

// @filename: loader.mjs
export async function resolve(specifier, parentModuleURL, defaultResolver) {
    const resolved = defaultResolver(specifier, parentModuleURL);
    const {url} = resolved;

    /**
     * Implement you own criteria here. For instance, you could use a glob pattern:
     * src/**\/*\/*.js - consider all js files in the "src" folder to be ES modules.
     */
    if (resolved.format === 'cjs' && url.endsWith('.esm.js')) {
        // Tell node to load this module as ES module even though it doesn't
        // have the "mjs" extension.
        return {url, format: 'esm'};
    }

    return resolved;
}

// @filename: foo.esm.js
import bar from './bar.esm'

// @filename: bar.esm.js
export default 42;

// Usage: node --experimental-modules --loader ./loader.mjs foo.esm.js

This way you can avoid patching TypeScript and it works without any third party libraries.

@SMotaal

This comment has been minimized.

Copy link

SMotaal commented Apr 26, 2018

Certainly, but performance and complexity makes it ideal for certain applications early on but not as requirements started to deal with integrations with other packages.

@que-etc

This comment has been minimized.

Copy link

que-etc commented Apr 26, 2018

Yeah, it's meant to be used only inside of an application. In case of a library, you'd definitely need to change the extension back to .mjs before publishing.

But anyway, with the former approach, integration with other packages is my biggest concern as well.

dtinth added a commit to dtinth/dt.in.th that referenced this issue May 6, 2018

@ctsstc

This comment has been minimized.

Copy link

ctsstc commented Jun 18, 2018

The future cannot be soon enough. Thanks for all the hard work on this. It definitely is hard to commit to something when it's "experimental".

@gfmio

This comment has been minimized.

Copy link

gfmio commented Jul 29, 2018

I have been needing this functionality for my own projects for quite a while. I had a script that would do the necessary transformations after the output had been generated, but it was a bit brittle, so I decided to fork the typescript package and patch it to support the generation of .mjs files (Github, NPM).

The patch itself was surprisingly easy since I only had to add a compiler option and the new extension type (Commit)

You can find my patched version as ts-mjs on NPM.

With the package installed as a dev dependency, you can now call the patched version of tsc as tsc-mjs and if you can add the compiler option --mjs to emit .mjs files instead of .js files.

If you use the --sourceMap compiler option, this will also emit the corresponding .mjs.map source map for your .mjs file.

If the output target for JSX would be preserve via one of the compiler options, regular JSX code with the regular extension .jsx will be emitted.

For example, if you run

tsc --sourceMap --declaration --target es3 --module commonjs index.ts

this will emit

index.d.ts (the declaration file)
index.js (the CommonJS ES3 module)
index.js.map (the source map for the CommonJS module)

If you run

tsc-mjs --sourceMap --declaration --target esnext --module esnext --mjs index.ts
index.d.ts (the declaration file)
index.mjs (the ES module)
index.mjs.map (the source map for the ES module)

The declaration files that are generated will be identical in both cases.

You can use this to publish hybrid modules by running both commands in succession (or in parallel, since they won't interfere with each other). The output will be:

index.d.ts (the declaration file)
index.js (the CommonJS ES3 module)
index.js.map (the source map for the CommonJS module)
index.mjs (the ES module)
index.mjs.map (the source map for the ES module)

If you have other source files, the corresponding 5 files will be generated for them.

If someone enters your package now via an .mjs file, node will automatically search for .mjs file in all import (or require) statements (and it will fall back to .js should an .mjs file not exist).

In your package.json, you can make the main field "dynamic" by removing the extension of the linked file.

{
    "main": "index",
    "browser": "index.js",
    "module": "index.mjs",
    "types": "index.d.ts"
}

For a real-life example of this being in use, check out my hsluv-ts library (Github, NPM).

(Note: The link to that NPM package will be broken during the next 24 hours, because I made a mistake while publishing, had to unpublish it and now I have to wait for 24 hours to republish it).


This should be straightforward to integrate into the mainline compiler, if desired, since the patch is so small. It just comes with the addition of one compiler option and it makes everyone's life a lot easier. Let me know what your thoughts are and I'm happy to create a pull request!

-@gfmio

@weswigham

This comment has been minimized.

Copy link
Member

weswigham commented Jul 29, 2018

This is your daily reminder that .mjs and how cjs/esm interop works in node is unstable and liable to change until it ships unflagged - you should not use it in a production workflow unless you're amenable to significant churn. 💓

We will support it in TS once it's stable - use what you find at your own risk. 😃

@Jamesernator

This comment has been minimized.

Copy link

Jamesernator commented Aug 1, 2018

@weswigham I would like to point out that regardless of what Node.js decides to do it's already possible to use any extension in the browser. Which I think is important.

Where I work most of the code is still primarily classic scripts, and I've been introducing bits of modules in internal tools, but to prevent tooling break from new files that are modules and to minimize any nightmarish configuration .mjs was a natural pick. NOTE that this is primarily browser based code (but there are some node tools too using @std/esm) and I think it would be good for TypeScript to support an extension that's guaranteed to be a module for this reason.

EDIT: Just to clarify the actual extension doesn't matter, but adopting modules into an existing code base of primarily classic scripts without breaking tooling is just difficult without a different extension. It's not necessarily important to support every extension but having at least one that's guaranteed to be a module source text rather than classic script makes introducing them into existing codebases a lot easier.

@demurgos

This comment has been minimized.

Copy link
Author

demurgos commented Aug 1, 2018

Regarding Typescript and Node's native ESM, there is an issue in nodejs/modules. It's relatively old but it wasn't posted here.

Changing the file extension of the output file is one of the least problems. On the emit side, TS tries to keep the differences minimal between the output using different module types. Node having different mechanism when consuming CJS or ESM may be a hard difference to hide.
The biggest issue though is consuming ESM and properly typechecking the dependencies. The type declaration files are mostly using ES syntax, even if the module is actually written in CJS (for example, it "lies" about available named exports). When using esModuleInterop, Typescript cannot know if the import statement is actually valid (ES namespace or default export). If you throw in dual builds and import-based resolution (for example .js/.mjs or main/module), the situation quickly becomes complicated: the type of the dependency depends on how you consume it.


This also a reminder to watch the nodejs/modules repo. The discussions may be hard to follow, but at least they're in a single place.


I'm currently looking into code coverage for native ESM in Node. It looks promising but is still experimental. I'll report back when things settle a bit.

@romap0

This comment has been minimized.

Copy link

romap0 commented Dec 7, 2018

Any news on it?

@saschanaz

This comment has been minimized.

Copy link
Contributor

saschanaz commented Dec 7, 2018

@romap0 TC39 proposed an alternative: https://github.com/tc39/proposal-modules-pragma

TypeScript will probably follow TC39 when a consensus happens.

@jpike88

This comment has been minimized.

Copy link

jpike88 commented Dec 7, 2018

I bit the bullet and used https://github.com/fuse-box/fuse-box works good

@demurgos

This comment has been minimized.

Copy link
Author

demurgos commented Dec 7, 2018

@saschanaz
My understanding is the the module pragma has been rejected.

@jpike88
Isn't fuse-box a bundler? I am not sure how this relevant to TS emitting .mjs (which is mostly a Node concern).

@romap0
We are still waiting for Node to settle on a design for ESM support. Discussions are advancing on the nodejs/modules repo.

@saschanaz

This comment has been minimized.

Copy link
Contributor

saschanaz commented Dec 7, 2018

@demurgos It's still in stage 1 proposal list: https://github.com/tc39/proposals#stage-1

@jpike88

This comment has been minimized.

Copy link

jpike88 commented Dec 7, 2018

I use fuse-box for my server code too (i also noticed it initialises a lot faster than the .mjs method, likely due to sub-optimal linking in Node itself)

@demurgos

This comment has been minimized.

Copy link
Author

demurgos commented Dec 7, 2018

@saschanaz
It's been in stage 1 for a few years now. It has been discussed at TC39 meetings and rejected. Have there been TC39 meetings discussing it again in the last months?

On the TC39 side, the most recent development (that I am aware of) has been around maybe adding a new module record type for interop with CJS.

@saschanaz

This comment has been minimized.

Copy link
Contributor

saschanaz commented Dec 7, 2018

It's added to the list 3 months ago so I would say it's not "rejected", as any rejected proposals instead go to "inactive proposals" list.

@revmischa

This comment has been minimized.

Copy link

revmischa commented Jan 15, 2019

My use case: developing a MJS/ES2015 module and importing it into another project. While I develop, I am running tsc -w. If a post-compilation step is needed to rename the output file to .mjs then how I can develop a project that depends on my library being recompiled with tsc -w?

Not a rhetorical question, I don't get it.
Isn't this how people normally write modules? Have another project that imports the module (via yarn link) and compile both projects in --watch mode? How can this work with ES2015 modules if a post-compilation step is required to rename a file from .js to .mjs? Am I missing something?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.