Encouraging users to use experimental features to reach better outcomes

[SMotaal]

Both --experimental-modules and --experimental-vm-modules have proven invaluable not only to Node's own implementation efforts, but to the community at large. The real benefits however can be limited if the typical day-to-day developer tooling has to be omitted when experimenting due to incompatibilities that may be remedied under similar experimental flags.

I think it is important for any project to have a clear direction when it comes to releasing experimental support, more specifically when it comes to tooling and interoperability.

[ljharb]

While it’s certainly great to get feedback, I think it is very dangerous to actively encourage people to use experimental features. Anyone who needs encouragement to use an experimental feature is very likely to not understand the inherent dangers of doing so.

[SMotaal]

I guess encouraging does imply making it too easy. How about making it at least possible instead…

Personal Account:

Since --experimental-modules I have struggled to work with my go-to-editor because it complained about things when the extension is not .js.

I used to author everything in .ts and used ts-node and things were working great. Sometimes I used code that was written as ES6 modules and that was .js and that was okay, as long as I used babel-node and configured things differently. Then @std/esm kind of made things a lot smoother.

With --experimental-modules I now needed to use .mjs and I was happy to ditch all those transpilation stages because while I used to author ES6, I was never really evaluating ES2015 modules before, but now I can. So to experiment, I need to get rid of all those things that round-trip my code from ES6-like to a non-Module intermediary then to a true ES2015+ module. Otherwise, it is not really experimenting with modules.

With that same notion where I want to be experimenting on the thing itself and not a piped approximation of it, I have spent almost a year waiting for upstream changes to the tooling I am used to. Since I was observing very closely, I appreciated more than everyone the amount of work involved in adding support for ES modules. I also realized that the idea of --experimental-modules and to be blunt about it .mjs created this shared desire to curb the enthusiasm, because while it is technically impossible for a tool to start supporting shipping ES modules while they don't yet exist, it is however not too much to ask for experimentally not breaking existing implementations on something as small as a byte aka m.

This left us with the following:

• Either experiment with modules the old way
• Or experiment with modules but exclude any use case you are familiar with and best able to provide feedback on for the community.

Did I not do the effort? Let's just say that the PR's are there but the amount of work it takes to patch a stale PR with new releases that continue to add essential tooling features like import.meta is mind-boggling because code change forces you to start from scratch.

So I cannot really provide the community with much about experimental modules aside from the fact that I believe that they are not ready for use case feedback from the community, unless they are ready to reset how they go about everything else.

Hope this alternative perspective can highlight the deeper aspects of the issue, not the polarizing headline.

[devsnek]

no matter what extension we end up using (and please get comfortable with the idea of mjs, its already been drafted to ietf) people are going to have to make changes.

add essential tooling features like import.meta is mind-boggling because code change forces you to start from scratch.

maybe i'm misunderstanding you but we already have import.meta, and since typescript isn't javascript i'm not surprised that you had to make a lot of changes.

[ljharb]

I see no problem with the implication that it’s currently very difficult to get useful experience with the experimental feature - this is largely because after 3-4 years of working on it, we decided to start the information gathering process from scratch with this working group. I don’t think we need implementation experience right now - what we need to focus on is agreeing on the use cases we will support/prioritize, and the features those necessitate, and then we can talk about what implementations would provide them, and then build one - and then we can worry about seeking feedback.

[benjamingr]

Since --experimental-modules I have struggled to work with my go-to-editor because it complained about things when the extension is not .js.

What editor? Sounds like something the editor should fix - should be simple enough to PR in :)

with new releases that continue to add essential tooling features like import.meta is mind-boggling because code change forces you to start from scratch.

Well, "real" ESM modules aren't ready yet - that's the sad truth. People are working on fixing that (you included!) but the fact some stuff is missing is a fact and we need to fix that before modules are "ready" for prime time.

I don't want to encourage users to use it for stuff other than experimentation until then and existing tools like @std/esm are a good stop gap in my personal opinion.

Hope this alternative perspective can highlight the deeper aspects of the issue, not the polarizing headline.

Definitely helpful 👍

[GeoffreyBooth]

@SMotaal Thanks for your personal anecdote. I think it’s very useful for stories like yours to be part of the conversation, to keep the user experience front and center.

In #42 I argued that Node should remove its current --experimental-modules implementation. I still feel that way today, especially after reading your experience. The longer that --experimental-modules remains out there, distributed, the more it seems like that implementation or something largely like it will end up shipping with Node as the final product.

In particular, I would be very surprised if .mjs ships in the final, unflagged version of modules. With respect to @devsnek, the reaction from the community to .mjs has been quite negative, with a strong desire from many users to avoid the ecosystem disruption that comes from a new file extension. And .mjs just isn’t needed; the NPM implementation and nodejs/node#18392 both demonstrate modules implementations that don’t require .mjs.

[ljharb]

The reaction isn't really relevant; there's use cases that necessitate "a file extension", and there's just no way we'll ever obtain consensus on any solution that doesn't include that. Both of those examples achieve "not needing .mjs" by disregarding/marginalizing/ignoring critical use cases of some of the members of this working group.

[SMotaal]

I really want to avoid making this another .mjs thread... If needed, we can just each find one of those and read it and come back when it is out of our system.

My initial thought behind this thread was to let everyone express how much --experimental-modules has or could improve the process of having the tool or tools smoothly transition to the eventual non-experimental modules, especially how much feedback from their users can enrich (or hinder) both the mutual outcome for all of us and their own development and transition efforts.

Opting to support or hold back is a call that each tool makes independently, but for users it only takes one tool holding back to either make it not possible to experiment or to have to do so by dropping that tool altogether and maybe not just for experimentation when predictability is questioned.

[SMotaal]

maybe i'm misunderstanding you but we already have import.meta, and since typescript isn't javascript i'm not surprised that you had to make a lot of changes

If I read the responses I get across issues and feature requests over at TypeScript, I can clearly see that there is a very strong force within the team to keep the syntax (aside from obvious type aspects) on par with ES, refusing adding new features or upgrading old features that are not compliant.

Although I was once all the way into TypeScript sugaring, like many, with the promotion of all the new ES features by the community, we have reached a point where at least for me, I just need to strip out type annotations (and maybe just affix the paths). Some of us are now experimenting with fast type-stripping without any deeply rooted overhead of a type checking engine at runtime (but the flux is making efforts very challenging).

If I were to venture a guess and hopefully I get corrected if I am wrong, TypeScript is now positioned well as a core development experience enhancing tool for pure JavaScript separate from mixed and pure TypeScript projects. At the very least, .d.ts files and inferred types are expected to be a factor when publishing to NPM for next few years.

Those are the kind of things that require consideration in a package-centric module system, and experimented on by those who are in a position to do so.

[SMotaal]

@GeoffreyBooth about locking out experimental features, can you elaborate on your view a little bit? I realize experimental kinda of throws curveballs all over the place and wonder if you have an alternative take on how to find and refine all the edge cases?

[MylesBorins]

I'm a huge -1 on removing the implementation we currently have and I highly doubt the technical project would support a move like that While individuals may not be satisfied with some of the UX, large portions of the implementation have nothing to do with file extensions. This thread feels like it is veering in a very unproductive direction. I'd like to advise scoping it to a desired outcome or closing it.

…

On Tue, Jun 12, 2018, 7:35 AM Saleh Abdel Motaal ***@***.***> wrote: @GeoffreyBooth <https://github.com/GeoffreyBooth> about locking out experimental features, can you elaborate on your view a little bit? I realize experimental kinda of throws curveballs all over the place and wonder if you have an alternative take on how to find and refine all the edge cases? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#122 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAecVya1bwAlsC4Q2Dn0fyGYPCwgMAH4ks5t76dzgaJpZM4UXMWD> .

[SMotaal]

I am for closing... I was hoping a far more constructive path, but I guess everyone reads a little less of what the other may be implying so trying to elaborate seems to create more miscommunication instead, not productive is the most suitable conclusion 👍

[GeoffreyBooth]

I’m not sure why @SMotaal and @MylesBorins are calling this discussion unproductive, but I assume it’s because I brought up .mjs and that felt like a sidetrack? I apologize for the confusion. I mentioned .mjs only as an example of something that might change before modules support gets released unflagged. Hey, maybe it won’t. Surely, however, some of the user-facing details of Node’s modules implementation will differ between the current experimental-modules and whatever modules implementation gets released unflagged. How to import CommonJS, for example, has had a lot of discussion: import.meta.require, via a loader, not at all, etc. But again, that’s just an example. I’m not trying to get into a rabbit hole of specific features, or argue for or against particular features or implementations; my point is just that from our group’s discussions it seems like some user-facing things will change, potentially many user-facing things.

@MylesBorins makes a good point that there’s lots of non-user-facing parts of experimental-modules. There’s certainly no good reason to remove those. So let me amend my suggestion: I think we should remove the user-facing parts of experimental-modules from released versions of Node, until we have a new implementation that this group is happy with and is considering shipping unflagged.

People will start using features that are released, even if they’re flagged, and I don’t see the benefit of the community experimenting with an API that seems likely to change, perhaps significantly. Or put another way, there’s no point in including experimental-modules in any released versions of Node unless you want people to use it, even if only for experimentation, and I don’t see why we would want that until we have an implementation we’re considering releasing as is—in other words, an implementation that we want to put out there as a public beta.

[devsnek]

@GeoffreyBooth seeing people use the current implementation has been (at least for me) quite useful. people aren't using it for their server deploys, they're messing with it on weekends for fun. i see at least three or four people messing with it every week and talking to them about their expectations, what worked, what didn't, etc has been fantastic. i think if we were able to connect to that and continue evolving the current design asynchronously from this group, we might really be able to connect the user story and the implementation soundly when the time comes.