support ES6 features

[clausreinke]

Several valuable ES6 features appear to be stable (arrow functions, classes (modulo private members), block scoping) and this month's (May 2013) tc39 meeting seems to be the deadline for a wiki-complete module system spec. Transpilers and engines are already prototyping support, and it is important that tern is not left behind.
If I recall correctly, the esprima and spidermonkey folks have started to agree on the syntax/AST extensions, but tern also needs to take the scoping, typing, and semantics into account (eg desugaring class syntax to ES5 features while keeping ES6 locations).

[marijnh]

Yes, this is a project that's on the horizon. It might be huge, and will be tricky as long as the features are non-final and poorly documented. I'm not going to work on it yet, but if you want to, the first thing would be to add an ES6 mode to Acorn that parses at least the stable parts of the new syntax.

[RReverser]

Which is to be closed by acornjs/acorn#110.

[marijnh]

@RReverser note that this is the Tern repository, and your PR adds ES6 support to Acorn. There's going to be a bunch of additional work to be done to teach Tern to understand the new constructs.

[RReverser]

Sure, my reply was rather for

the first thing would be to add an ES6 mode to Acorn that parses at least the stable parts of the new syntax.

[bvalosek]

What would be the next steps for getting ES6 support in Tern now that acorn is parsing the es6 features?

I'd like to contribute in some way if possible, or even put a bounty for this feature.

[RReverser]

There is still need to fix template literals in loose parser which is somewhat more tricky, but in general parser-side is ready.

[tomitrescak]

Great news and a great job! How can we get it to work? I am using Sublime 3 with TernJS plugin which for now somehow marginally works with ES6 while the tern_for_sublime crashes. Thanks!

[marijnh]

@tomitrescak Tern does not work with ES6 yet, only Acorn (the parser) does.

How does tern_for_sublime crash (i.e. what code, what error message)? It shouldn't -- theoretically it should just ignore the things it can't understand.

[mk0x9]

Is there any roadmap to support ES6 features?

I'm especially interested in classes and module imports.

[angelozerr]

Is there any roadmap to support ES6 features?

Tern starts supporting ES6 with ecma6.json https://github.com/marijnh/tern/blob/master/defs/ecma6.json

@marijnh has managed Promise, but ecma6.json must be improved again like Iterator support.

But I think Acorn should be customized to support ES6 inside tern.

[RReverser]

@angelozerr

But I think Acorn should be customized to support ES6 inside tern.

I don't really understand why it should be customized to support ES6 differently inside Tern? How is that different from already implemented ES6 support?

[angelozerr]

@RReverser I don't know acorn, but with acorn you can set ecmaVersion http://marijnhaverbeke.nl/acorn/#section-4 although inside tern it is never done.

I suppose setting ecmaVersion gives features according the version (perhaps acorn is able to support iterator protocal?

But perhaps I'm wrong?

[marijnh]

Yes, we are currently not enabling ES6 parsing when calling acorn.parse because Tern does not understand the ES6 node types. The way to start supporting this would be to extend the AST walkers in infer.js so that they properly handle the new syntactic constructs.

I will to work on this eventually, but I'll need time and funding. The second can probably be addressed with another crowd-funder, which I intend to start at some point when I am less busy.

[RReverser]

@marijnh I was also just thinking about getting to work on this as soon as finish better loose support in JSX, as I've got some free time, but same problem with 2nd part of your statement :)

[Oceanswave]

[marijnh]

Here's what you can do: Open a bounty on Bountysource for the specific ES6 features you are most interested in. Link it from this issue, and either put money into it right away, or ask me and/or Ingvar what we'd charge to implement it. There's a quite a lot of new stuff in ES6, but if we split it up into smaller features, estimating the cost and raising the funds shouldn't be too hard.

[simonzack]

👍 I've been using sublime code intel which doesn't support ES6. If ternjs supported ES6 I would definitely switch, as it will make coding much more pleasant. I see class support as being the most important here, as it will give the most completions compared to other features.

[marijnh]

Hi all,

I've spent the past weeks implementing ES6 support for Tern, including module support. You can see a demo at http://ternjs.net/es6/doc/demo/#ES6

Since giving work away and hoping for the best isn't turning out to be a good way to earn a living, I'm holding the code hostage (not open sourcing it) until the Bountysource item at https://www.bountysource.com/issues/1141202-support-es6-features reaches $2000. If you want to donate towards that sum in another way, contact me.

Some notes on the module loader system:

• The node module loader was split out of the node plugin and into modules (basic module tracking) and commonjs (require-style module loading) plugins.
• There's a new es_modules plugin that manages import/export module connections, also using the infrastructure in the modules plugin
• The modules plugin only defines a very primitive module resolution strategy (resolve relative paths). The node plugin adds node's module resolution rules. So if you are using import for node-style modules (under node_modules/...), you have to load the node plugin to find them.
• Plugins dependencies will be mostly automatically loaded, but if you use Tern in a way where plugins can't be automatically loaded (raw script tags, for example), you'll have to include them manually.

Other ES6 stuff implemented in this branch:

• Block scopes and let/const bindings
• Destructuring and default values in assignments, parameters, and bindings
• Tuples/heterogenously typed arrays to make patterns like [a, b] = [foo, bar] work
• Rest arguments
• Generators
• Symbols and symbol properties
• For/of iteration
• Classes and super calls
• Extended object literal notation (computed properties, succinct methods, name-only properties)
• Arrow functions
• Template strings
• All methods and types added by the ES6 spec
• Improved existing support for Promises

[marijnh]

Update: Since BountySource doesn't seem to do Euros, I've switched the goal's currency to dollars. (I.e. you just got a 10% off deal).

Also, everybody who contributes $30 or more has my permission to start using this code by git cloning http://marijnhaverbeke.nl/git/tern-es6/. Proper packaging will happen when the goal amount is reached.

[gilligan]

@marijnh the linked tern-es6 version does not yield any results for anything when I try it with the tern vim plugin. I always only get the emptry string as return value. Have you tested it with the vim plugin ? Are there any kind of breaking changes that would need to be adapted on the vim plugin side ?

[marijnh]

Hi @gilligan, did you npm install after getting/upgrading the code? I just tried with Vim, and it works on my machine. (The HTTP protocol, which the Vim plugin uses, stayed the same.)

[L42y]

@marijnh after I append export default to any file, tern-es6 seems to stop working anymore

[gilligan]

@marijnh sometimes it works, apparently rarely though. I will try to work out some minimal test cases

[marijnh]

@L42y @gilligan The export default issue is fixed now (was a typo in a property name)

[L42y]

@marijnh Where I can get the patch? Couldn't find it on GitHub or http://marijnhaverbeke.nl/git/tern-es6/

[marijnh]

Ah, right, I'm in the middle of a server migration and marijnhaverbeke.nl isn't switched over yet. I updated the repo on that server as well now.

[paulvi]

@marijnh asking for bounties is great life check for project,

but what will be after $2000 mark?
well, I don't think I would cover your time spent, and what for next work to do?

Should it be actually some monthly check to motivate (but not cover all expenses)
What can be next?

I guess http://ternjs.net/ should get "Donate" button or this bounty will stay open forever.
Bountysource looks better as more transparent.

P.S. it is midnight now.

[gilligan]

I would have to agree. I have already contributed 30USD because that was what I was supposed to pay to for allowance to use the es6 branch. That already felt somewhat "off" to be honest.

As you can see I have no issues with contributing something to an open source project, but this should happen organized in a transparent fashion in my opinion. Currently this is the exact opposite.

ES6 support is already there and people have to pay to use it. That is just weird. At this point you might as well make it a closed source project and ask me to just buy it. Which I would in fact do (or already did).

But you can't have your cake and eat it too.

Please don't take this the wrong way, I am merely trying to point out how this is perceived by me and maybe others. It remains your project with which you can do as you please of course.

Best,
Tobi

Sent from my iPhone

On 30 Aug 2015, at 18:01, Paul Verest notifications@github.com wrote:

@marijnh asking for bounties is great check,

but what will be after $2000 mark?
well, I don't think I would cover your time spent, and what for next work to do?

Should it be actually some monthly check to motivate (but not cover all expenses)
What can be next?

I guess http://ternjs.net/ should get "Donate" button or this bounty will stay open forever.
Bountysource looks better as more transparent.

P.S. it is midnight now.

—
Reply to this email directly or view it on GitHub.

[spion]

Just for the record, I donated and I don't perceive it that way; I would definitely not donate for a closed source project, because I believe that the benefit an open source Tern would bring to the JS community now that we are getting ES6 can be truly great (ES6 as a language is a lot more amenable to static analysis, but as its new few editors have decent support for it which makes people reluctant to switch).

A closed source Tern would not stand a chance of doing that though.

[marijnh]

We've reached $ 1,772 at bountysource and I've recently received another Tern-related contribution outside of that platform that I'm going to count too, putting us over the goal. I've just merged my es6 branch into master, and the ES6 work is now under the MIT license.

@gilligan Could you elaborate on your concerns about lack of transparency? I stated exactly what I did, and what happens when my financial goal is reached. You state that you'd prefer for this to be a closed-source commercial project. How would not being able to read, modify, or fork the source benefit you?

[marijnh]

(Please open new issues for shortcomings and bugs found in the ES6 implementation.)

[marijnh]

And I forgot to mention -- there'll be a new release with this code tomorrow.

[Swatinem]

Thank you for your hard work and for making tern “open source”.
But I just can’t hold back my opinion that people should be weary of non-copyleft licenses and CLAs, because they allow such “un-sportsmanlike” behavior.

[marijnh]

@Swatinem Do elaborate what exactly you are perceiving as un-sportmanlike behavior. (And how a copyleft license would have helped.)

[JustinTulloss]

@marijnh It may be in everybody's best interest to lock this thread as it's now a discussion of licenses instead of ES6 support in tern.

[gilligan]

Sorry, I obviously failed at expressing myself. I would certainly not prefer a closed source approach. I have been travelling the whole day and I am deas tired. Will reply properly soon, sorry & g'nite :)

Sent from my iPhone

On 30 Aug 2015, at 22:04, Marijn Haverbeke notifications@github.com wrote:

We've reached $ 1,772 at bountysource and I've recently received another Tern-related contribution outside of that platform that I'm going to count too, putting us over the goal. I've just merged my es6 branch into master, and the ES6 work is now under the MIT license.

@gilligan Could you elaborate on your concerns about lack of transparency? I stated exactly what I did, and what happens when my financial goal is reached. You state that you'd prefer for this to be a closed-source commercial project. How would not being able to read, modify, or fork the source benefit you?

—
Reply to this email directly or view it on GitHub.

[bennidhamma]

I think Marin should be a slave to all of us and have no rights, only
obligations to JavaScript programmers who don't have the time or skill to
do what he can.

Seriously, thanks Marin. I think getting the community to sponsor the work
is an incredibly reasonable approach. The money hopefully enables you to
have some time to really work on this project and then it's shared with the
community in true open source fashion. Honestly, I almost like the hostage
approach more - less risk as a donater.

On Sun, Aug 30, 2015, 3:19 PM Tobias Pflug notifications@github.com wrote:

Sorry, I obviously failed at expressing myself. I would certainly not
prefer a closed source approach. I have been travelling the whole day and I
am deas tired. Will reply properly soon, sorry & g'nite :)

Sent from my iPhone

On 30 Aug 2015, at 22:04, Marijn Haverbeke notifications@github.com
wrote:

We've reached $ 1,772 at bountysource and I've recently received another
Tern-related contribution outside of that platform that I'm going to count
too, putting us over the goal. I've just merged my es6 branch into master,
and the ES6 work is now under the MIT license.

@gilligan Could you elaborate on your concerns about lack of
transparency? I stated exactly what I did, and what happens when my
financial goal is reached. You state that you'd prefer for this to be a
closed-source commercial project. How would not being able to read, modify,
or fork the source benefit you?

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#116 (comment).

[gilligan]

Since the trolling is now commencing I will move this elsewhere. You should think about how you communicate. About exaggerations and accusations that abort discussions instead of resolving them by actually having them.

On 31 Aug 2015, at 04:17, Ben Joldersma notifications@github.com wrote:

I think Marin should be a slave to all of us and have no rights, only
obligations to JavaScript programmers who don't have the time or skill to
do what he can.

Seriously, thanks Marin. I think getting the community to sponsor the work
is an incredibly reasonable approach. The money hopefully enables you to
have some time to really work on this project and then it's shared with the
community in true open source fashion. Honestly, I almost like the hostage
approach more - less risk as a donater.

On Sun, Aug 30, 2015, 3:19 PM Tobias Pflug notifications@github.com wrote:

Sorry, I obviously failed at expressing myself. I would certainly not
prefer a closed source approach. I have been travelling the whole day and I
am deas tired. Will reply properly soon, sorry & g'nite :)

Sent from my iPhone

On 30 Aug 2015, at 22:04, Marijn Haverbeke notifications@github.com
wrote:

We've reached $ 1,772 at bountysource and I've recently received another
Tern-related contribution outside of that platform that I'm going to count
too, putting us over the goal. I've just merged my es6 branch into master,
and the ES6 work is now under the MIT license.

@gilligan Could you elaborate on your concerns about lack of
transparency? I stated exactly what I did, and what happens when my
financial goal is reached. You state that you'd prefer for this to be a
closed-source commercial project. How would not being able to read, modify,
or fork the source benefit you?

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#116 (comment).

—
Reply to this email directly or view it on GitHub.

[marijnh]

@gilligan I was trying to have a discussion, by asking you to clarify your issue. I see no exaggeration or accusation in any of my messages here. I am, in fact, rather pissed off about your tone earlier in the thread, but I am trying to figure out what you mean before jumping to any conclusions.

[gilligan]

Oh i did not mean you at all. And if I have offended you in any way I am sorry since this was not my intention.

What I was referring to were messages by people that jumped in drawing black/white pictures while apparently not understanding what I was on about.

"OMG This looks to me like someone said something about F/OSS that I don't like. Lets jump in !"

I have used open source software all my life, make anything l do open and try to open source things at work.

What I was trying to say was...

1. i thought it as unfortunate that there is some donation button with no clear goal/timeline/topic. You clarified it in comments later on

2. i found it odd that you would ask for donations on something that already exists in a private repo. I practically bought my es6 tern copy.

That being said I did donate and started talking to my employer about supporting the project. It boils down to me (and some colleagues) finding the process a bit odd.

I pointed out that it is your project and you can do with it as you wish. Yet I think it is only fair that I'd be allowed to share some concerns.

Lets not draw conclusions too quickly and cool down.

On 31 Aug 2015, at 09:54, Marijn Haverbeke notifications@github.com wrote:

@gilligan I was trying to have a discussion, by asking you to clarify your issue. I see no exaggeration or accusation in any of my messages here. I am, in fact, rather pissed off about your tone earlier in the thread, but I am trying to figure out what you mean before jumping to any conclusions.

—
Reply to this email directly or view it on GitHub.

[marijnh]

2. i found it odd that you would ask for donations on something that already exists in a private repo. I practically bought my es6 tern copy.

The strategy of asking people to pay for something they can get for free isn't usually terribly effective (though I'm definitely working on that front too, see my funding page, also linked from the project page). I've found that having some kind of 'carrot' that makes donating attractive can make a huge difference. I do not see how implementing first, then asking for money, is problematic. At least that way you're able to play with the implementation and verify that it works before deciding on whether to shell out for it.

[gilligan]

The difference is exactly the difference between donating and purchasing.

By "donating" 30 USD I purchased a license (by shape of your expressed allowance) to use the es6 featured branch in your private repo.

I did do this because I am really fond of tern - it is a productivity boost to me - but I find this quite odd.

That is the reason why I suggest that you might want to think about a different model if money is a key factor. I have no problem in actually purchasing it. You could still have it on github and/or make it free for non commercial use.

So nobody pushed the donate button? Yes, that is what the open source community is like a lot. But your carrot is not just a carrot it hints at the fact that maybe you want/need a different model.

So that is what I referred to with the cake analogy. A github repo won't make an income.

On 31 Aug 2015, at 12:38, Marijn Haverbeke notifications@github.com wrote:

2. i found it odd that you would ask for donations on something that already exists in a private repo. I practically bought my es6 tern copy.

The strategy of asking people to pay for something they can get for free isn't usually terribly effective (though I'm definitely working on that front too, see my funding page, also linked from the project page). I've found that having some kind of 'carrot' that makes donating attractive can make a huge difference. I do not see how implementing first, then asking for money, is problematic. At least that way you're able to play with the implementation and verify that it works before deciding on whether to shell out for it.

—
Reply to this email directly or view it on GitHub.

[marijnh]

it hints at the fact that maybe you want/need a different model.

No, I definitely don't. See also this blog post

[Swatinem]

I really don’t want to make this into a flame war of any kind.
Also note that I’m not a lawyer so I probably have no idea what I’m talking about.

To reply to @gilligan

I pointed out that it is your project and you can do with it as you wish.

I don’ think that statement is true. According to github its the shared work of 59 different contributors (although clearly @marijnh has done most of it)

But the statement is also true because of the license. The MIT License basically gives anyone the ability to do whatever they wish with the software, including making a closed source fork and sell it for money.

Doing so is just fine by the license, but it sure offends some of the other 58 contributors. It sure would have pissed me off a lot. But alas, I am not one of the contributors.

I actually did check out the code some time ago and started implementing a feature without much success and never got around to actually finish it and make it work. But now at least I know not to contribute anything in the future. (Unless I sub-license my fork as GPL, which the MIT license allows me to do)

I admire your goal to actually make a living writing open source software and I wish you all the best for it. As in, doing the things you love as opposed to doing the things someone else tells you to. But the point of open source is that it invites others to contribute as well. And holding [their] code hostage as you said is simply not a good way to invite further contributions.

Again, I’m very sorry if this might have started a flame war. I will just shut up from now on :-)

[marijnh]

I did not hold anyone's code hostage but my own. The master branch was entirely unaffected by the crowd-funding, and the only new work on the (briefly) non-free branch was my own work.

I'm going to lock this topic since I really don't have the emotional energy to deal with this kind of bullshit.