Backward compatible ESM support

[nbouvrette]

Describe the feature you'd like to request

Since Next.js 11.1.3-canary.69, ESM externals has been turned on by default using the experimental: { esmExternals: true } in this commit.

One of the challenges is that the module resolutions between CommonJS and ESM is different. From Node documentation:

The current specifier resolution does not support all default behavior of the CommonJS loader. One of the behavior differences is automatic resolution of file extensions and the ability to import directories that have an index file.

By having this, it means that you could end up with different import paths depending on if you use CommonJS or ESM which will be a really bad developer experience. On top of this, to provide the correct support for Next.js, modules would have to be backward compatible since not all packages support ESM today.

Describe the solution you'd like

It looks like the way to do would be to support --experimental-specifier-resolution=node but from looking at previous issues (here and here), it does not seem to work with Next.js (not sure why).

I was not able to get it to work myself, so maybe we could have this as a new experimental option that is supported by Next.js?

Describe alternatives you've considered

I tried to build a backward compatible package using this article but ended up with clunky paths which degraded the developer experience.