Enable Node.js to run with Microsoft's ChakraCore engine #4765

Closed
wants to merge 9 commits into
from

Projects

None yet
@kunalspathak
Contributor

(Note from the CTC (Fishrock123): This thread is expected to garner a lot of attention. Comments that are not productive to discussing the technical aspects may be removed.)


Enable Node.js to optionally build and run on Microsoft's ChakraCore JavaScript engine.

We implemented a V8 API shim (aka chakrashim) which takes advantage of ChakraCore runtime hosting APIs (JSRT). This shim implements most essential V8 APIs so that the underlying JavaScript engine changes are transparent to Node.js and native addon modules written for V8.

Here is the summary of commits :

  • chakrashim source code
  • ChakraCore v1.1.0.1 source code
  • build script changes to build Node.js with ChakraCore on Windows OS for x86, x64 and ARM
  • node-gyp changes to build Node.js with ChakraCore on Windows OS for x86, x64 and ARM 1
  • gyp changes to to build chakrashim and ChakraCore on Windows OS for x86, x64 and ARM 2
  • node source code changes needed to run on ChakraCore
  • unit test updated for ChakraCore

1 node-gyp changes won't directly be submitted via PR, but will be first submitted to upstream branch as pointed out in nodejs/node-gyp#873. They are included in PR so reviewers can build and test Node.js with chakracore. Node-gyp changes are divided in two commits, first is based on changes needed to build node.js with chakracore for x86 and x64 for Windows OS and other based on changes need to support building it for ARM architecture for Windows OS.

2 gyp changes are also divided in two commits. One for x86/x64 and other for ARM.

kunalspathak added some commits Jan 12, 2016
@kunalspathak kunalspathak chakrashim: implement V8 APIs on Chakra
ChakraShim implements essential V8 APIs needed by Node.js on top of the
Chakra runtime hosting API (JSRT). This enables Node.js and other native
addon modules written for V8 to build and run with the Chakra JavaScript
engine.
77a5f8d
@kunalspathak kunalspathak deps: chakracore source code
Source code of [chakracore](https://github.com/Microsoft/ChakraCore.git)
that lights up Node.js for Chakra.
Building Node.js with Chakra produces chakracore.dll along with other binaries
that is needed by node.exe to function.
8a8c98c
@kunalspathak kunalspathak referenced this pull request in Microsoft/ChakraCore Jan 19, 2016
Closed

[Question] Integration to node.js #140

@Fishrock123
Member

@kunalspathak Thanks for the PR! This has a very large impact so please expect review to take a considerable amount of time. :)

@daekano
daekano commented Jan 19, 2016

VERY cool! Don't envy the code reviewers here though!

@joshmanders

This is by far one of the coolest things I've seen happen to Node.js since the convergence of io.js and Node.js.

@MikeFielden

Legit. Thanks, cant wait until this happens 👍

@unicodeveloper

The review will really take a lot of time and scrutiny! 🍴

@SilverIgniter

+1️⃣ guys, really good job! 💥

@robcolburn robcolburn commented on the diff Jan 19, 2016
common.gypi
@@ -10,6 +10,8 @@
'component%': 'static_library', # NB. these names match with what V8 expects
'msvs_multi_core_compile': '0', # we do enable multicore compiles, but not using the V8 way
'python%': 'python',
+ 'node_engine%': 'v8',
@robcolburn
robcolburn Jan 19, 2016

Nit: is node_engine established elsewhere? Maybe js_engine? This line and everywhere else for consistency

@indutny
indutny Jan 19, 2016 Member

We usually just prefix node stuff with node_ in here.

@ingIor
ingIor commented Jan 19, 2016

This is incredibly cool, I have a request.

Please do not post comments on the GH issue unless you have something important to add. These issues gain a lot of attention and it makes it incredibly hard for collaborators to communicate.

Locking the issue to collaborators means other people from the outside who have a significant ***** contribution or want to help can't do that.

Comments like +1 -1 and such create a significant amount of noise.

Support open source, keep the discussion clean.

EDIT(@trevnorris): Possible accidental use of inappropriate word? Has been removed.

@trevnorris
Contributor

@kunalspathak What happens when v8 needs to be updated and the API doesn't match the Chakra shim? Will we need to hold off until the shim also has an update?

@jasnell
Member
jasnell commented Jan 19, 2016

@kunalspathak ... thank you for submitting this. As you can imagine, it's going to be a big review and could take some time to settle out.

@trevnorris ... I have separately reached out to each of the V8 and Chakra teams and invited both to sit down face to face to work through the API/ABI impact of this change and figure out how we can make the ABI layer more robust. I'm working out the logistics for that face to face now and want to make sure to extend the invite to all the @nodejs/ctc members as well. There are a ton of questions this brings up and I think sitting down for an afternoon to hash things out would be quite productive.

@benjamingr benjamingr and 3 others commented on an outdated diff Jan 19, 2016
@@ -1101,6 +1112,10 @@ def configure_intl(o):
pprint.pformat(icu_config, indent=2) + '\n')
return # end of configure_intl
+def configure_engine(o):
+ engine = options.engine or 'v8'
@benjamingr
benjamingr Jan 19, 2016 Member

Prefer function default parameter values def configure_engine(o = 'v8'):

@ChALkeR
ChALkeR Jan 19, 2016 Member

But it's not a function parameter.

@pritambaral
pritambaral Jan 19, 2016

Then default options.engine at nodejs/node-chakracore@12cc5d9#diff-e2d5a00791bce9a01f99bc6fd613a39dR380

    dest='engine',
+   default='v8',
    help='Use specified JS engine (default is V8)')
@kunalspathak
kunalspathak Jan 19, 2016 Contributor

@pritambaral , @benjamingr - Default is v8 engine as seen in vcbuild.bat

@pritambaral
pritambaral Jan 19, 2016

@kunalspathak Yes, but the line commented-upon here is in configure and not vcbuild.bat. It would be better if the or-set in this line were replaced with the very first assignment of options.engine, which is what I think @benjamingr intended.

@kunalspathak
kunalspathak Jan 19, 2016 Contributor

Fair point.

@jasnell
Member
jasnell commented Jan 19, 2016

Marking it as semver-major for now (just to be safe), although semver-minor might be appropriate once we get further along in the review. Very happy to see this tho.

@benjamingr benjamingr commented on the diff Jan 19, 2016
deps/chakrashim/core/CONTRIBUTING.md
@@ -0,0 +1,54 @@
+## Contributing Code
@benjamingr
benjamingr Jan 19, 2016 Member

Is there any need for these files here?
Why isn't ChakraCore brought as a dependency?

@MrRio
MrRio Jan 19, 2016

It's because this is the shim, the Core is brought in as a dep, but this is closely linked to node.

@kunalspathak
kunalspathak Jan 19, 2016 Contributor

Thats right. The sources in core folder are from Microsoft/ChakraCore. It is just easier to update the chakracore sources in node deps when they are aligned with their origin in github counterpart.

@panuhorsmalahti
panuhorsmalahti Jan 19, 2016

Why not use a git submodule?

@jianchun
jianchun Jan 19, 2016

Why not use a git submodule?

This follows the convention of existing deps -- source copy instead of submodule. It allows more flexibility such as making changes here before upstream when necessary (Not familiar with submodule, don't know if that's also supported).

@MylesBorins
MylesBorins Jan 19, 2016 Member

@panuhorsmalahti node currently does not use submodules for dependencies. It may be a good idea to document how dependencies are updated.

Currently for dependencies such as v8, libuv, or npm we take version updates as a single commit updating the state of the current directory, generally from a collaborator of the project we are downstream from. In between large updates we will sometimes backport changes as individual patch's.

@panuhorsmalahti
panuhorsmalahti Jan 20, 2016

@jianchun: okay, I didn't know about that convention. I disagree with it, but I guess changing that convention is another discussion. To answer your question, if the upstream needs to be modified without changing e.g. the master branch, a new branch should be created upstream.

@sam-github
sam-github Jan 21, 2016 Member

@panuhorsmalahti git submodules don't allow the thirdparty source to be patched, or work with thirdparty deps that are not available via git.

@Qix-
Qix- commented Jan 19, 2016

Maybe this is a stupid question, but what is the advantage to this PR? To use Chakra instead of V8, right? Is that a runtime specification or a build specification?

@ChALkeR
Member
ChALkeR commented Jan 19, 2016

Am I correct that this currently works only on Windows platform? Initial Linux support is still on the roadmap of ChakraCore: https://github.com/Microsoft/ChakraCore/wiki/Roadmap. And that is decribed as «make it link, make it run, no JIT».

@sethgaurav

Yes, its currently Windows only. For cross-platform support, the key target for next six months on the roadmap is to get the interpreter & runtime working. JIT would come after that (don't read it as no JIT forever - it was just a breakdown of what we need enable and its ordering for next 6 mos).

@ChALkeR
Member
ChALkeR commented Jan 19, 2016

Will ChakraCore be treated as a first-class citizen? That is:

  1. Will there be binary builds with ChakraCore for each Node.js release provided from the nodejs.org site?
  2. Would it be guaranteed that ChakraCore must work with every next Node.js version, or would it be ok if v6.2.0 builds with ChakraCore, v7.0.0 breaks with ChakraCore (let's say the shim update didn't get in in due time), and v7.1.0 works again with ChakraCore?
  3. What if v7.3.0 breaks again with ChakraCore in a semver-minor version, because of some additive change that didn't get yet ported to the shim?
@mikeal
Member
mikeal commented Jan 19, 2016

@ChALkeR I think those questions can't be answered until after this PR lands. We have support in Node.js for architectures that we don't have in CI and pretty much all the ones we do have were added some time after support for the architecture was landed. I don't think anyone wants to add another vector to CI until we know there is some stability here and we're only going to figure that out after it lands.

@jasnell
Member
jasnell commented Jan 19, 2016

@ChALkeR ... at least in my opinion, it depends on what is meant by "first-class citizen"

  1. For the very near term, likely not. Having officially supported builds of Node+ChakraCore is likely quite some time off and wouldn't be able to happen at least until the cross-platform work is done. Similar to how we (IBM) handled our Node-on-PPC and Node-on-AIX builds, it would be more likely that Msft would host their own downloads with builds of Node+ChakraCore (please correct me if I'm wrong @orangemocha or @kunalspathak), at least until a decision is made to officially provide Chakra support.

2 and 3: I think it's too early to determine that. Given the dependency on the V8 APIs currently, and given how quickly those could change, there will need to be dedicated effort by Microsoft and other interested collaborators to ensure that the Chakra shim continues to work. It is entirely possible that semver-major update of V8 could land that could break these. Until we're sure that the shim is stable, we likely won't want to commit to ensuring that things would continue to work.

+1 to what @mikeal just said as I was typing this :-)

What I'd personally like to see is getting this landed but clearly indicating that it is experimental and unsupported for the time being.

@ChALkeR
Member
ChALkeR commented Jan 19, 2016

@mikeal @jasnell You mean that the initial answer is «no» to all questions (i.e.: no guarantees at all for the time being), with a possibility to change in the future? That seems legit.

@mikeal
Member
mikeal commented Jan 19, 2016

Yup, this is sort of a classic "let's put this behind a flag and see what happens" situation :)

@Fishrock123
Member

Yup, this is sort of a classic "let's put this behind a flag and see what happens" situation :)

As a note, this is not a runtime feature, so it is explicitly behind a build-time flag.

@vkurchatkin
Member

It seems like a nightmare to support.

Imagine the following: someone make a PR that uses v8 API. CI breaks on Chakra build, because shim doesn't support said API. What we are going to do? wait until Chakra implements them? abandon the PR?

What if next week someone PRs support for JSC or SpiderMonkey?

@ChALkeR
Member
ChALkeR commented Jan 19, 2016

@vkurchatkin If I read the answers above by @mikeal and @jasnell correctly — ignore the breakage and land, but mention the people who will support ChakraCore and give them a chance to fix this.

@Fishrock123 Fishrock123 added the windows label Jan 19, 2016
@jasnell
Member
jasnell commented Jan 19, 2016

@vkurchatkin ... initially, we would not gate acceptance of any V8 changes based on whether or not it breaks Chakra. The responsibility would be on those collaborators who are supporting the experimental Chakra support to update that code to work again.

@mikeal
Member
mikeal commented Jan 19, 2016

Imagine the following: someone make a PR that uses v8 API.

The general direction the project wants to go in is to become more vm agnostic, which means using less of the API, not more, and moving towards a neutral API that all vm's can support natively. That's a long way off, we have a lot of work to do, but this is the first step in that direction.

@vkurchatkin
Member

@ChALkeR well, the goal is to make it officially supported one day, obviously. Otherwise committing this code doesn't make anything. So, when we decide that it is supported, what would be the answer then?

@vkurchatkin
Member

The general direction the project wants to go in is to become more vm agnostic, which means using less of the API, not more, and moving towards a neutral API that all vm's can support natively. That's a long way off, we have a lot of work to do, but this is the first step in that direction.

This seems to be a step in the opposite direction, really. I mean, it implements v8 API, not neutral API. Also, adding another vm into source tree seems like a step in the opposite direction as well.

@TooTallNate TooTallNate and 1 other commented on an outdated diff Jan 19, 2016
@@ -51,6 +53,37 @@
],
},
+ 'conditions': [
+ ['node_engine=="v8"', {
+ 'target_defaults': {
+ 'defines': [
+ 'NODE_ENGINE="<(node_engine)"',
@TooTallNate
TooTallNate Jan 19, 2016 Contributor

This could probably be moved to a common "defines" array, since its duplicated 11 lines below.

@kunalspathak
kunalspathak Jan 19, 2016 Contributor

Sure.

@Qix-
Qix- commented Jan 19, 2016

@jasnell

What I'd personally like to see is getting this landed but clearly indicating that it is experimental and unsupported for the time being.

Then it should go on a dedicated branch, not master.

@mikeal
Member
mikeal commented Jan 19, 2016

Then it should go on a dedicated branch, not master.

That's just not in line with how we've added any experimental features before. Branches might be used from time to time to develop a feature but it's never used by the public until it lands in master. For instance, AsyncWrap has been experimental and in master for quite a while.

@jbergstroem
Member

Before we start running this on our CI -- @kunalspathak do you have output of vcbuild.bat test somewhere?

@Qix-
Qix- commented Jan 19, 2016

@mikeal Sounds exactly like what a change like this deserves.

@Fishrock123
Member

@Qix- Note: this does not replace V8, it only gives a built-time option to.

Edit @joshmanders, nope, sorry!

@joshmanders

@Fishrock123 was that meant to be directed at me?

@indutny
Member
indutny commented Jan 19, 2016

I'm slightly concerned about download size of our repository. Having a big chunk of code living there and not being used in 99% of cases seems to be a huge waste of bandwidth. (I know many people clone the repo, or get the source from tarballs)

@jasnell
Member
jasnell commented Jan 19, 2016

@vkurchatkin ... it's a step in the right direction, it's not the only step :-) The next step is to begin moving towards a more abstracted ABI between Node core and the VM layer, which is something that ought to make a LOT of things in core easier... for one, we'd be able to take V8 updates faster than we currently do, even potentially landing new V8 majors within stable branches. It's a long road, however, and there are many steps along the way that still need to be figured out.

@jasnell
Member
jasnell commented Jan 19, 2016

@indutny ... I do share the concern over download size. One possible approach is to handle this similar to how we handle the ICU dependency. IF the config option is set to use this, then the build goes out and grabs the tar file and drops it into the deps folder, then and only then is this large chunk of optional code downloaded.

@mikeal
Member
mikeal commented Jan 19, 2016

If we're considering pulling in stuff during build time we need to ping @nodejs/build :)

@naholyr
Contributor
naholyr commented Jan 19, 2016

Many APIs and/or options didn't come to exist for the sake of platform equality (thinking about sendfile for example), is this about to change or are you going to wait for Linux/Mac compatibility of Chakra?

@vkurchatkin
Member

@jasnell I'm really trying to see how it makes transitioning to abstract VM API easier, but I can't. I clearly see how it will make maintaining node much more difficult in short term, though.

@nathanpeck nathanpeck commented on the diff Jan 19, 2016
deps/chakrashim/core/bin/ch/WScriptJsrt.cpp
+ fileName = L"script.js";
+ fileNameLength = wcslen(fileName);
+ if (argumentCount > 3)
+ {
+ IfJsrtErrorSetGo(ChakraRTInterface::JsStringToPointer(arguments[3], &fileName, &fileNameLength));
+ }
+ returnValue = LoadScript(callee, fileName, fileNameLength, fileContent, scriptInjectType);
+ }
+
+Error:
+ if (errorCode != JsNoError)
+ {
+ JsValueRef errorObject;
+ JsValueRef errorMessageString;
+
+ if (wcscmp(errorMessage, L"") == 0) {
@nathanpeck
nathanpeck Jan 19, 2016

Brace style is not consistent here

@Fishrock123
Fishrock123 Jan 19, 2016 Member

@nathanpeck Please point comments about Chakracore files to https://github.com/Microsoft/ChakraCore -- there's not much we are going to do about it here. :)

@jasnell
Member
jasnell commented Jan 19, 2016

I'm not seeing how it makes maintaining node more difficult in the short term as maintaining support for the Chakra shim would not be in the critical path of any other changes. For the majority of collaborators, in the short term, they could simply ignore the Chakra stuff and would be perfectly fine. There would be zero impact. If we made it a download-on-build option then it would be even less of an impact.

In the long term, it helps making the transition to an abstract VM API easier because the Chakra team has had to go through this exercise of getting Chakra to transparently act like V8. That means they've gathered a significant amount of experience about what it would take to create that abstraction. By combining their experience and bringing the various teams together for discussion, we can begin to identify how that abstraction needs to be defined.

@davidmurdoch

I haven't seen anyone answer why this should be added.

Why should node support the ChakraCore engine? What are the benefits of doing so?

@mscdex
Contributor
mscdex commented Jan 19, 2016

+1 to making this downloadable on demand at build time like the intl data.

@orangemocha
Member

+1 to what @jasnell said:

initially, we would not gate acceptance of any V8 changes based on whether or not it breaks Chakra. The responsibility would be on those collaborators who are supporting the experimental Chakra support to update that code to work again.

... but hopefully when and if the Chakra team has proven their ability to keep up with those breaks, we'll be able to include support in stable releases. I guess we will cross that bridge when we get there, and it's a good idea to deem this as 'experimental' for now.

We can still add support for ChakraCore in CI, so that at least we know when things break, even though it wouldn't prevent acceptance of PRs. I will get working on that.

In terms of reviewing this change, I am tempted to say that we could treat the stuff under deps/ as a black box. The remaining delta is actually fairly small.

@kunalspathak it might make sense to maintain ChakraShim in its own repo, and update it here as a whole.

@vkurchatkin
Member

I'm not seeing how it makes maintaining node more difficult in the short term as maintaining support for the Chakra shim would not be in the critical path of any other changes. For the majority of collaborators, in the short term, they could simply ignore the Chakra stuff and would be perfectly fine.

I'm talking about period after that.

In the long term, it helps making the transition to an abstract VM API easier because the Chakra team has had to go through this exercise of getting Chakra to transparently act like V8. That means they've gathered a significant amount of experience about what it would take to create that abstraction.

So, they've already done that. If this code remains in a separate repo, their experience is not going to become any less useful.

@jbergstroem
Member

I'm not very comfortable landing anything that has 51 test fails on the currently only supported OS, be it optional or not. I'll look at running the PR against our own CI shortly.

@kunalspathak
Contributor

@jbergstroem - We are aware of these test failures and most of the unit test failures are because of unimplemented features in chakrashim. We are working on getting them to 0 failures. We will keep this thread posted on our status.

@loopmode

@davidmurdoch chakra currently has a more complete es6/es7 implementation. Of course once v8 catches up it'll be a different story.

@kevva
kevva commented Jan 19, 2016

As a package maintainer, I'm afraid that I'll have to make sure that stuff works across two engines, in addition to all node versions. While having another engine has its upsides, it's most likely going to cause compatibility issues.

@Fishrock123
Member

@kevva I think many of us have similar concerns. We'll be trying to make sure everything is as compatible as possible.

@kevva
kevva commented Jan 19, 2016

@Fishrock123, yeah, but it's almost out of nodes scope in a way. Even though everything in the node API works across both engines, we still have to account for JS features that will differ between them. Not to mention all the native abstractions.

@kobalicek

Maybe @obastemur (jxcore) can contribute to the discussion?

I would like to see something like zero overhead Neutral API that can target more JS engines, I would myself contribute to such effort. I even started developing mine called JSNI (js native interface) after V8 broke several times my own addon, but I was never able to support more than V8 and it is based on macros.

@Qix-
Qix- commented Jan 19, 2016

@Fishrock123

Note: this does not replace V8, it only gives a built-time option to.

I understand that. What I'm saying is this is a major change that not many people will need/use but many, many will have to support and maintain. I don't see this as a plus for Node at this time, personally.

@jagtesh
jagtesh commented Jan 19, 2016

As much as I'd like to see this feature in, I'm afraid that some of the finer differences between V8 and Chakra fragment the community. Packages supporting Chakra only may emerge, while some other existing packages may not work on Chakra. Their use together will create a mess on both engines.

We could mitigate this (somewhat) the Ruby gems way by allowing packages to specify what engines they support. Still feel some thought needs to go into this from the more experienced members of our community.

@kobalicek

@jagtesh I think the question is about how to support more engines transparently. V8 doesn't have stable API, maybe introducing a stable and neutral API can take less time than continuing using V8 in long term. Also, having neutral API would mean that all engines are considered equal.

@mikeal
Member
mikeal commented Jan 19, 2016

Fragmentation.

The differences between Chakra and V8 in terms of ES-NEXT support are roughly the same as the differences in V8 we have been shipping in different major releases so there's no reason to think that Chakra is going to cause fragmentation on the JS layer and because they are shimming the V8 API they won't be fragmenting native API users either.

@megastef

Question: does it mean that v8 specific native modules like gc-stats or memwatch-next would run on Chakra? Are Chakras gc metrics correct and useful for monitoring ?

@lightningtgc

Any performance comparison between V8 and Chakra in Node?

@kunalspathak
Contributor

@lightningtgc - Please refer to this blog that talks about performance comparison.

@yjhjstz
yjhjstz commented Jan 20, 2016

Node.js addon like node-inspector can't be used anymore. :( @kunalspathak
How to handle this?

@kunalspathak
Contributor

@yjhjstz - Yes, we have not implemented debugger support for node+chakracore yet. You can follow its development at JsRTDebugging.

@GoddessIvy

Different engines have different API. Do we have a starderd API document?

@denghongcai

JavaScript Engines like V8, Chakra, may be best to unify the ABI layer.

@irfn
irfn commented Jan 20, 2016

What would this mean with respect to ES6 features. Would one expect the same level as Edge.

Would this mean that node will be built with both engines and one can select on runtime(command line switch etc.) or would you have separate prebuilt binaries with node and v8

@obastemur
Contributor

What would this mean with respect to ES6 features. Would one expect the same level as Edge.

Yes. Looks like this PR comes with Async/Await feature is enabled. (nodejs/node-chakracore@77a5f8d#diff-7da8fe3a685013a34f9825ff00f2f343R64)

Details on async/await -> https://blogs.windows.com/msedgedev/2015/09/30/asynchronous-code-gets-easier-with-es2016-async-function-support-in-chakra-and-microsoft-edge/

@jasnell
Member
jasnell commented Jan 20, 2016

As far as ES features are concerned, until a decision was made to support Chakra officially, the baseline for supported features would have to be whatever is supported by V8. Anything supported by Chakra but not supported by V8 would be "use at your own risk" and would not be something could be supportable by core.

@shigeki
Contributor
shigeki commented Jan 20, 2016

@kunalspathak This PR includes two new features. One is adding ChakraCore as a new JS engine and the other is adding a new os and arch support where Node running Windows on arm. Is it difficult to separate this into two PRs? I'm not sure V8 can be built and run on the Windows arm.

@Qard
Member
Qard commented Jan 20, 2016

We have a github org just for nodejs stuff now, and public repos are free...maybe we should just start pulling stuff out of deps and use git submodules that point to our own forks we can maintain with whatever patches we need.

Also, I really hope we can get this merged if for nothing else than just to have a control sample in our attempts to formalize a consistent internal API. It's easy to overlook the rough edges in our internals when only interfacing with one VM that seems to work okay most of the time, even though its API is actually really weird compared to most other VMs.

@orangemocha orangemocha referenced this pull request in nodejs/build Jan 20, 2016
Open

Managing optional dependencies #310

@orangemocha
Member

use git submodules

git submodules would be nice in principle but in practice git submodule implementation is more trouble that it's worth. Anyway, I opened nodejs/build#310 so that we can continue the discussion about optional dependencies there.

@mazhlekov

nice, next step - bundle app in .exe :D

@miketaylr miketaylr referenced this pull request in OpenSourceSystemPodcast/projects Jan 20, 2016
Closed

Episode 9 #36

@Fishrock123
Member

@mazhlekov That's out of scope for this. :)

@kunalspathak
Contributor

@irfn, to answer your questions:

Would one expect the same level as Edge.

Not necessarily. ChakraCore might have ES6 features not yet present in Edge because ChakraCore binaries get ultimately merged to Edge and not vice-versa. So you will get latest ES6 features in node+chakracore that might not be released in Edge yet.

Would this mean that node will be built with both engines and one can select on runtime(command line switch etc.) or would you have separate prebuilt binaries with node and v8

No. Currently, you can build node either with chakracore or v8 and not both. Command line to build with chakracore is

vcbuild.bat ia32 chakracore
@kunalspathak
Contributor

Is it difficult to separate this into two PRs?

Thanks @shigeki for feedback. I could separate the PRs. From your comments, I suppose this has to be divided in 3 PRs however:

  1. Node-gyp changes that needs to go in node-gyp repo.
  2. Make node run on Windows for ARM a new PR.
  3. Enabling Node.js with Chakracore engine (the original PR).

For this to happen, I will have to overwrite commits (by taking it 1 and 2 off from some commits) and update my branch. Are you fine with that?

@arrowrowe arrowrowe referenced this pull request in dyweb/web-stuff Jan 20, 2016
Closed

Weekly for 2016/01/27 #36

6 of 8 tasks complete
@zchrykng

I don't have much skin in this game, other can being a user of node, but really like the idea of moving toward a standardized API/ABI for the javascript engines. Node's dependence on V8 was always one of the things that made me most nervous about the platform.

@shigeki
Contributor
shigeki commented Jan 21, 2016

@kunalspathak Yes, I'm fine with it. Reading the minutes of ctc meeting today, this discussion will be continued in an issue. You should not be in a hurry to work on my requests.

@trevnorris
Contributor

@zchrykng The @nodejs/api group has done some preliminary research into an abstraction layer. Unfortunately since core relies on v8 specifics it's difficult to abstract at that layer. So as a more feasible goal I think we've decided to first tackle the native user exposed API. That unfortunately won't help this case.

@rvagg
Member
rvagg commented Jan 21, 2016

An interesting prospect that came out of discussion around nodejs/LTS#62 (specifically how to enable non-ABI breaking V8 upgrades during Stable cycles) from @bnoordhuis was to start with a shim that reflects the V8 API at a point in time and let V8 evolve behind that shim while keeping it stable for Node's purposes. The idea here was that we could do this at the start of a Stable cycle, V8 could do it's ABI-breaking dance but we get to keep the same ABI (and API) as long as we can manage those changes behind a shim of their earlier API.

One could take that approach a bit further and suggest that we it with a longer-term in mind and evolve the shim to something resembling a slightly more engine-agnostic layer that can be backed with V8, ChakraCore or another engine. That way we would have a concrete starting point (the V8 API at the time we started) and take an evolutionary approach to making it more ideal. IMO this would work a lot better than the revolutionary approach of defining an entire API from scratch and doing the implementation work as this is almost begging to be a dead-in-the-water project unless there are serious and dedicated resources applied to such an effort.

@Fishrock123
Member

One could take that approach a bit further and suggest that we it with a longer-term in mind and evolve the shim to something resembling a slightly more engine-agnostic layer that can be backed with V8, ChakraCore or another engine. That way we would have a concrete starting point (the V8 API at the time we started) and take an evolutionary approach to making it more ideal. IMO this would work a lot better than the revolutionary approach of defining an entire API from scratch and doing the implementation work as this is almost begging to be a dead-in-the-water project unless there are serious and dedicated resources applied to such an effort.

Any reason we wouldn't use this to start developing our own more agnostic public-facing API? It seems like a good starting point where we can feel engine differences and similarities.

@vkurchatkin
Member

Any reason we wouldn't use this to start developing our own more agnostic public-facing API? It seems like a good starting point where we can feel engine differences and similarities.

V8 is a terrible choice for that. JSC, Chakra and SpiderMonkey API have a lot in common and V8 is very different.

@MartinJohns

I would suggest to close further discussions on this PR until nodejs/roadmap#54 brought up results.

@rumkin
Contributor
rumkin commented Jan 26, 2016

@MartinJohns But how does nodejs/roadmap#54 actually relate to discussion. ChakraShim is not an ABI. And if we talking about Chakra than it should sounds like "Do we need a zoo of engines".

@rvagg
Member
rvagg commented Jan 26, 2016

You're not really adding value here @trendzetter and your comments are starting to get insulting. If you have trouble moderating yourself then we can help you with that.

Same goes for anyone else having trouble being constructive in this thread; this is not your personal blog and there are people trying to get things done and make objective decisions around here and we could do without the silly ranting.

@rvagg
Member
rvagg commented Jan 26, 2016

@trendzetter thankyou, your concerns are noted, along with others who have been expressing similar and opposing views on this particular topic. There is obviously a lot of complex context that feeds into any decision around this pull request.

However, please, let's draw a line under discussion regarding Microsoft, many of us have heard most of these arguments and are aware of both extreme points of view. Continuing to repeat them here will only contribute to you being further marginalised from the only debate that actually matters here and will have no impact on the end decision. It's all been said already.

Thankyou for your input, however.

@rvagg rvagg locked and limited conversation to collaborators Jan 26, 2016
@rvagg
Member
rvagg commented Jan 26, 2016

We've decided to lock this thread, for a few days at least, to put a damper on the unproductive chatter before it gets fully out of control.

If you'd actually like to engage in productive discussion regarding Node.js being tied to V8 or becoming something closer to VM neutral, then please head over to nodejs/roadmap#54. Please try and stay civil and if you don't have anything helpful to add, just observe (there's a Subscribe button on the side).

kunalspathak added some commits Jan 28, 2016
@kunalspathak kunalspathak build: build with chakrashim/chakracore
Enable building Node.js with chakracore engine for windows OS - x86, x64
and ARM.

* Configure to build Node.js with "v8" (default) or "chakracore" JS engine
   with optional vcbuild.bat switch.
* chakrashim uses js2c with a namespace.
* Configure msvs_use_library_dependency_inputs to export symbols
   correctly (otherwise functions not used by node.exe but might be
   needed by native addon modules could be optimized away by linker).
* Add parameter `arm` to `vcbuild.bat` that will pass `--openssl-no-asm`
* Introduced `msvs_windows_target_platform_version` to be used to set
   Windows SDK for ARM.
b6e81a2
@kunalspathak kunalspathak node-gyp: Support to build with chakracore engine
Add support to allow node.js build with chakracore enigne for Windows OS
x86/x64 architecture.

* Enables building native addon modules for Node.js with multiple
   node-engines.
* Configure msvs_use_library_dependency_inputs to export symbols
   correctly (otherwise functions not used by node.exe but might be
   needed by native addon modules could be optimized away by linker).
81fb6d5
@kunalspathak kunalspathak gyp: Support to build with chakracore engine
Add support to allow node.js build with chakracore enigne for Windows OS
x86/x64 architecture.

* Configure msvs_use_library_dependency_inputs to export symbols
   correctly (otherwise functions not used by node.exe but might be
   needed by native addon modules could be optimized away by linker).
c1ae676
@kunalspathak kunalspathak node-gyp: Support to build with chakracore for ARM
* Support building node.js with chakracore on Windows on ARM
* Building chakracore for ARM has a dependency on Windows SDK installed
on the machine. Update python script to populate
`WindowsSDKDesktopARMSupport` and `WindowsTargetPlatformVersion`
for ARM builds by configuring msvs_windows_target_platform_version. This
will be used in `chakracore.gyp` to pass as parameter to `msbuild.exe`.
6e97a0b
@kunalspathak kunalspathak src,lib: Node source changes for Chakra engine
* Add runtime property "process.jsEngine" to indicate current JS engine
   node runs on (JS engine is determined at build time).
* Update REPL to work with Chakra engine.
* Buffer/SlowBuffer currently does not support @@species constructor.
   Mark them as such to avoid failure as per ES6 TypedArray @@species spec.
* Minor fix in `string_bytes.cc` because of way Chakra handles
   `String::NewExternal`.
abccd71
@kunalspathak kunalspathak gyp: Support to build with chakracore for ARM
* Support building node.js with chakracore on Windows on ARM
* Building chakracore for ARM has a dependency on Windows SDK installed
on the machine. Update python script to populate
`WindowsSDKDesktopARMSupport` and `WindowsTargetPlatformVersion`
for ARM builds by configuring msvs_windows_target_platform_version. This
will be used in `chakracore.gyp` to pass as parameter to `msbuild.exe`.
432d4a9
@kunalspathak kunalspathak test: Fixed unittest for Chakra engine
There were certain unittest that depends specifically on v8 internals and/or
engine specific error message. This change adds if-else to expect different
error message for Chakra engine or skip running test case if it test v8
internals not present in Chakra or if it test features unimplemented by
chakrashim.

Modified test python script to take engine name depending on which it
will match the baselines for v8 vs. chakracore engine.
19b2e48
@orangemocha orangemocha unlocked this conversation Jan 28, 2016
@kunalspathak
Contributor

I have updated the changes in the PR to incorporate as much feedback as possible and restructured the commits. I have also updated the PR description to reflect the new commit ordering. Thanks!

@orangemocha orangemocha locked and limited conversation to collaborators Jan 28, 2016
@jasnell
Member
jasnell commented Jan 29, 2016

@kunalspathak +1, it may take a bit of time to review but I'll try to get some comments in by next week. Thank you for being patient!

@rvagg rvagg unlocked this conversation Jan 30, 2016
@rvagg
Member
rvagg commented Jan 30, 2016

Deleted comment by @runvnc. Please take it to your blog, this PR is for technical discussion at the PR at hand. Discussion about whether Node.js should even head down a multi-VM route are taking place at nodejs/roadmap#54. Off-topic rants should be taken to tumblr, wordpress, or some other forum where you can engage in focused discussion rather than this PR. Please have a bit of respect for the collaborators of this project.

@Qix-
Qix- commented Jan 30, 2016

@rvagg I thought it was a pretty valid point, though it does belong in the roadmap discussion. Suggesting legitimate concerns be taken to Tumblr is a bit crass, though.


/ontopic is there a reason this isn't being run against the automated tests in CI?

@MrRio
MrRio commented Jan 30, 2016

I haven't be paid by Microsoft for the record. 😂
On Sat, 30 Jan 2016 at 09:27, Josh Junon notifications@github.com wrote:

@rvagg https://github.com/rvagg I thought it was a pretty valid point,
though it does belong in the roadmap discussion. Suggesting legitimate
concerns be taken to Tumblr is a bit crass, though.


Reply to this email directly or view it on GitHub
#4765 (comment).

James Hall
Director
Parallax Agency Ltd
M. 07894950320

@jbergstroem
Member

@Qix-: Collaborators (inc me) have already run tests (see above). We're counting 50+ fails on Windows, Linux being worse and SmartOS, FreeBSD not working at all. I treat this PR as a door opener for nodejs/roadmap#54.

@hansdunk

@jbergstroem

We're counting 50+ fails on Windows, Linux being worse and SmartOS, FreeBSD not working at all.

Collaborators (inc me) have already run tests (see above).

This must be a joke. ChakraCore does not even compile on Linux yet...

@rvagg You have a bigger responsibility than deleting peoples' messages here. Starting with answering 'How come this PR can be considered seriously at this very point' ?

Forget about Microsoft is this or that. There are very good managers at Microsoft, and the bad ones. Like in any other place. The 'project management' behind this PR Has clearly failed. They supposed to wait, fix their issues and start the discussion first.

Node.js was supposed to be governed by community. Sadly, this PR shows otherwise.

@waynebloss

@hansdunk Deleting comments that are doing nothing more than speculating on Microsoft's motives is a pretty important job to do. We already have a good, objective, drama-free summary of the basic technical considerations that need to be made here (with some points clarified a few comments down) and discussion on those points has begun in other threads already.

Forget about Microsoft is this or that.

Exactly. I read the deleted comments if I in fact received them all in my email...they were both walls of text containing nothing more than rumors and pure speculation about Microsoft's business and marketing practices.

The 'project management' behind this PR Has clearly failed. They supposed to wait, fix their issues and start the discussion first. Node.js was supposed to be governed by community. Sadly, this PR shows otherwise.

Don't you think that's a bit dramatic? It's a Pull Request which has led to a discussion. It's not the end of Node.js being governed by its community. I'm not even sure how it's a failure on anybody at Microsoft's part other than the fact that it might be a bad idea to have a Chakra-specific shim in Node.js (for the record - I think it's a bad idea and that a non-Chakra specific ABI would be a better idea).

@MartinJohns

It's always hilarious when people claim some is bought by Microsoft or they're working against the community, just because their own opinion is not reflected. The community is large and this is a sensitive subject, there are people on both sides and on middle ground.

@waynebloss

@hansdunk Here's the discussion. We're in it. Sorry it didn't happen according to the exact timeline that you were expecting. :( Yes, it's still dramatic that you think Node.js community governance is over because one pull request (from a community member) happened before you thought it should.

Here's one of the technical bits that has been discussed here (among others): Do you think that it's a good idea for Node.js to have an ABI/facade for it's Javascript engine or should it be v8 only?

The core technical committee is having a discussion about this PR. Did you read the notes from their last meeting?

All you're doing is flinging accusations about and not really contributing anything at all. Please stop doing that.

@Qix-
Qix- commented Jan 30, 2016

Seeing as how not one collaborator has appeared to consider the negatives of this PR, it's understandable that people are a little uncomfortable.


@jbergstroem okay, but is there not a CI integration added to Github to be checking PRs automatically where test results can be viewed by the public?

@benjamingr
Member

@Qix- have you read the TC meeting notes or the issue and discussion about accepting a second engine? That discussion is held and a lot of people are expressing their opinion - they're just not doing it on the pull request itself.

@jokeyrhyme

Fun self-censorship exercise: swap "Microsoft" for "Google", and "Chakra" for "v8". If your sentence is still true then don't bother posting it.

@ghost
ghost commented Jan 30, 2016

My vote is also to wait for Chakra becoming reliable cross-platform product first, before considering this work.
Meanwhile, it would interesting if Gecko JägerMonkey (Firefox) guys are interested, who happen to have widest/maturest coverage when it comes to cross platforms!
Then there is v7 (https://github.com/cesanta/v7) for embedded systems (known for "whole JS engine in one C file!" v7.c) that will also bring value to the mix due to its unique platform support (imagine your washing machine or fridge running node.js over a lightweight JS engine without v8/Google'ism inside).

@rvagg
Member
rvagg commented Jan 31, 2016

We'd like to not have to re-lock this thread but will do so if necessary. Comments from a user who appears to have created a throw-away account simply for the purpose of posting derailing comments here have been removed.

I personally really hate deleting comments and I'm not even going to argue that there aren't legitimate discussion-worthy points being made in the comments being deleted. However, discussion that deviates significantly from technical discussion on this PR and that are intended to continue tangential argumentation will be removed. Please find a new forum for these posts, nobody wants to deny you your opinions, this just isn't the place for them and you're hurting Node.js in general by serving as a distraction from all of the productive things we could otherwise be doing.

@thefourtheye thefourtheye commented on the diff Jan 31, 2016
deps/chakrashim/core/Build/armasm.targets
+ SmallerTypeCheck ="%(ClCompile.SmallerTypeCheck)"
+ StringPooling ="%(ClCompile.StringPooling)"
+ StructMemberAlignment ="%(ClCompile.StructMemberAlignment)"
+ SuppressStartupBanner ="%(ClCompile.SuppressStartupBanner)"
+ TreatSpecificWarningsAsErrors ="%(ClCompile.TreatSpecificWarningsAsErrors)"
+ TreatWarningAsError ="%(ClCompile.TreatWarningAsError)"
+ TreatWChar_tAsBuiltInType ="%(ClCompile.TreatWChar_tAsBuiltInType)"
+ UndefineAllPreprocessorDefinitions ="%(ClCompile.UndefineAllPreprocessorDefinitions)"
+ UndefinePreprocessorDefinitions ="%(ClCompile.UndefinePreprocessorDefinitions)"
+ UseFullPaths ="%(ClCompile.UseFullPaths)"
+ UseUnicodeForAssemblerListing ="%(ClCompile.UseUnicodeForAssemblerListing)"
+ WarningLevel ="%(ClCompile.WarningLevel)"
+ WholeProgramOptimization ="%(ClCompile.WholeProgramOptimization)"
+ XMLDocumentationFileName ="%(ClCompile.XMLDocumentationFileName)"
+
+ TrackerLogDirectory ="%(ClCompile.TrackerLogDirectory)"
@thefourtheye
thefourtheye Jan 31, 2016 Contributor

Nit: from here onwards everything is not strictly in ascending order

@kunalspathak
kunalspathak Feb 1, 2016 Contributor

Thansk @thefourtheye for pointing it out. However since these sources are pulled from ChakraCore repo, feel free to open an issue on ChakraCore.

@jbergstroem
Member

@Qix- not yet since I'd have to fork this PR and add commits since it doesn't change vm by default. I opted for opening one of our windows slaves and testing it there instead.

@darth10
darth10 commented Feb 2, 2016

A fork sounds like a much better option IMHO.
On 02-Feb-2016 3:27 am, "Johan Bergström" notifications@github.com wrote:

@Qix- https://github.com/Qix- not yet since I'd have to fork this PR
and add commits since it doesn't change vm by default. I opted for opening
one of our windows slaves and testing it there instead.


Reply to this email directly or view it on GitHub
#4765 (comment).

@matthiasg

If Node were more open to other engines the auto build process could be configured to automatically produce more builds with different engines. As a totally independent fork would likely stay behind quickly and thus there would be no incentive to use it even.

As chakra being windows only. If some microprocessor comes out which were to require a modified version of v8 (e.g stripped down etc) I would consider this the same thing. Developers could use node everywhere, but there might be an alternative js engine under the hood which might work better on their platform. Whether they could use native modules is of course not necessarily guaranteed for all platforms. That is sadly already the case anyway.

Actually having chakra or any other engine in node would not be helpful. Fork with automated build seem more logical to me.

@isiahmeadows

@matthiasg The tone I drew from this (and some discussion elsewhere) is that this is yet another attempt to abstract Node from the runtime it uses. It's not unheard of, and it's even being done for Chrome extensions and the Java Runtime with Nashorn. I think it being an option in core (even if it means full vm module support is limited to one or more specific platforms) would just further legitimatize these efforts, as well as making future work far easier.

@Fishrock123 Fishrock123 pushed a commit to Fishrock123/node that referenced this pull request Feb 2, 2016
@vkurchatkin @rvagg vkurchatkin + rvagg test: remove Object.observe from tests
Testing this wasn't really useful, besides Object.observe is going to be
deprecated.

Also this test fails with Chakra (#4765) for obvious reason.

PR-URL: nodejs#4769
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
959e051
@rvagg rvagg changed the title from Enable Node.js to run with Microsoft's ChakraCore engine to Enable Node.js to run with Microsoft's ChakraCore engine Feb 3, 2016
@rvagg rvagg removed the ctc-agenda label Feb 3, 2016
@orangemocha
Member

At yesterday's CTC meeting, it was resolved to let these changes live in a separate repository under the nodejs/ organization. The rationale behind this decision (or at least my interpretation of it) is that we want to encourage experimentation with this project, yet at this time we don't want to put any additional burden on the collaborators of the main Node.js project to maintain the chakracore version. In particular, issues about the chakracore version should be directed to the new repo (name TBD). Of course, people interested in contributing are more than welcome to do so in the new repo. There is also a @nodejs/chakra team that can be mentioned within the nodejs org.

I suggest that we keep this PR open to continue providing feedback to the actual code changes, which I think would be welcome by the authors. Then instead of merging the changes into this repo, we can push them to the new repo.

@kunalspathak , can you add two changes to the README:

  1. How to build node with chakracore. This can be a part of nodejs/node-chakracore@b6e81a2
  2. A visible notice that tells users to file node-chakracore specific issues into the node-chakracore repo rather than nodejs/node. I would keep this as a separate commit.

As usual, please keep the discussion constructive.

cc: @nodejs/ctc

@jasnell
Member
jasnell commented Feb 4, 2016

I would also point out that it's possible that we could include the chakra
builds in CI and regular nightlies.
On Feb 4, 2016 6:25 AM, "Alexis Campailla" notifications@github.com wrote:

At yesterday's CTC meeting #5058,
it was resolved to let these changes live in a separate repository under
the nodejs/ organization. The rationale behind this decision (or at least
my interpretation of it) is that we want to encourage experimentation with
this project, yet at this time we don't want to put any additional burden
on the collaborators of the main Node.js project to maintain the chakracore
version. In particular, issues about the chakracore version should be
directed to the new repo (name TBD). Of course, people interested in
contributing are more than welcome to do so in the new repo. There is also
a @nodejs/chakra https://github.com/orgs/nodejs/teams/chakra team that
can be mentioned within the nodejs or g.

I suggest that we keep this PR open to continue providing feedback to the
actual code changes, which I think would be welcome by the authors. Then
instead of merging the changes into this repo, we can push them to the new
repo.

@kunalspathak https://github.com/kunalspathak , can you add two changes
to the README:

  1. How to build node with chakracore. This can be a part of Microsoft@
    b6e81a2
    nodejs/node-chakracore@b6e81a2
  2. A visible notice that tells users to file node-chakracore specific
    issues into the node-chakracore repo rather than nodejs/node. I would keep
    this as a separate commit.

As usual, please keep the discussion constructive.

cc: @nodejs/ctc https://github.com/orgs/nodejs/teams/ctc


Reply to this email directly or view it on GitHub
#4765 (comment).

@thefourtheye
Contributor

it was resolved to let these changes live in a separate repository under the nodejs/ organization

Will that repo also adhere to our Node.js CoC, Contribution, Moderation and other procedures we follow?

@jasnell
Member
jasnell commented Feb 4, 2016

Yes, it would fall under the same policies as the main repo. I imagine a
chakra team would be created and eventually a WG would emerge.
On Feb 4, 2016 6:44 AM, "thefourtheye" notifications@github.com wrote:

it was resolved to let these changes live in a separate repository under
the nodejs/ organization

Will that repo also adhere to our Node.js CoC, Contribution, Moderation
and other procedures we follow?


Reply to this email directly or view it on GitHub
#4765 (comment).

@orangemocha
Member

+1 for a WG to be formed around this, and keeping the policies in line with the main repo as much as possible. In the interest of relieving Node's collaborators from additional duties, we might want to make some slight modifications to the collaborator guide to replace "collaborators" with "WG members".

@mikeal
Member
mikeal commented Feb 4, 2016

Forming a WG

For organizational purposes this may make sense but there's one thing that troubles me about it. The goal of the WG will be to get this eventually merged into Core, if contentious issues come up in that repo it would be best to escalate those to the CTC, not just to the WG membership. This would avoid decisions that are controversial being made in that repo that will make it difficult to eventually merge the work into core.

@jasnell
Member
jasnell commented Feb 4, 2016

Agreed. Couldn't that be put into the WG charter tho? :-)

On Thu, Feb 4, 2016 at 10:48 AM, Mikeal Rogers notifications@github.com
wrote:

Forming a WG

For organizational purposes this may make sense but there's one thing that
troubles me about it. The goal of the WG will be to get this eventually
merged into Core, if contentious issues come up in that repo it would be
best to escalate those to the CTC, not just to the WG membership. This
would avoid decisions that are controversial being made in that repo that
will make it difficult to eventually merge the work into core.


Reply to this email directly or view it on GitHub
#4765 (comment).

@orangemocha
Member

Agreed. I was suggesting that we replace collaborators with WG members but still allow for CTC escalation for controversial issues. We should probably also promote some level diversity in the WG membership so that controversy can be had 😄

@jasnell
Member
jasnell commented Feb 4, 2016

I'd also suggest that changes in the repo be limited to only those directly impacting the implementation of the chakra support. Anything that can be upstreamed into the main repo should be.

@Qix-
Qix- commented Feb 5, 2016

The rationale behind this decision (or at least my interpretation of it) is that we want to encourage experimentation with this project, yet at this time we don't want to put any additional burden on the collaborators of the main Node.js project to maintain the chakracore version.


I would also point out that it's possible that we could include the chakra builds in CI and regular nightlies.

Perfect. This is what should be happening. Happy this is how it turned out 👍

@FezVrasta

Is it just me, or till ChakraCore is not ported to all the platforms supported by V8, all this stuff is completely useless?

I can't see how you could release a ChakraCore version of Node that works only on Windows, it would mean an hell of fragmentation...

@rvagg rvagg added a commit that referenced this pull request Feb 8, 2016
@vkurchatkin @rvagg vkurchatkin + rvagg test: remove Object.observe from tests
Testing this wasn't really useful, besides Object.observe is going to be
deprecated.

Also this test fails with Chakra (#4765) for obvious reason.

PR-URL: #4769
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
f7ed473
@orangemocha
Member

Update: we are in the process of transferring https://github.com/microsoft/node to nodejs/node-chakracore. Once that's done, I'll create an issue in that repo inviting collaborators to a newly formed node-chakracore WG, which will act as the repo collaborators.

In the meantime, if reviewers of this PR are comfortable giving it a 'LGTM as a starting point for the new repo', we can make the changes of this PR the head of the chakracore-master branch. Otherwise, we can resume the review process in the new repo.

@thefourtheye
Contributor

LGTM as a starting point for the new repo

I think I would be a little confident if we had a green CI run.

@orangemocha
Member

@thefourtheye assuming you mean on the regular CI, which doesn't test the chakracore build. Green status on the chakracore build is still work in progress.

https://ci.nodejs.org/job/node-test-pull-request/1606/ shows a merge conflict. @kunalspathak can you rebase the PR onto the current master?

@mikeal
Member
mikeal commented Feb 9, 2016

BTW, I don't think a straightforward "repo transfer" will work in GitHub because that repo is in the same fork network as one already in this repo. That means you'll have to essentially pull down the repo and all branches and push it to a brand new repo in this org. It's annoying, and you'll lose the issues, but I don't know of a way around it.

@orangemocha
Member

@mikeal, thanks for the suggestion. We were actually able to work around it but removing the fork link first (had to contact GH support for it).

@mikeal
Member
mikeal commented Feb 9, 2016

@orangemocha awesome! good to know that is an option, this will probably come in useful again in the future :)

@orangemocha
Member

On second thought, rebasing here is not necessary, as we can push to the new repo the master that this PR is up-to-date with, and deal with the conflict in the next master -> chakracore-master integration.

Here's a CI run with no rebasing: https://ci.nodejs.org/job/node-test-commit/2175/. It shows 2 failures on ARM (linux). Investigating whether these could be legit or flaky. /cc @nodejs/testing

@Trott
Member
Trott commented Feb 9, 2016

@orangemocha Those are both known flaky tests that are marked as flaky in current master. One of them has a fix ready to go in a PR that might land in another day or three. The other one is probably Flaky Test Public Enemy Number One once that PR lands.

@orangemocha
Member

Thanks for clarifying that @Trott ! I guess that means that the CI run looks good for this PR.

@Trott
Member
Trott commented Feb 9, 2016

@orangemocha Yes. The TL;DR of my comment is: Those are known flaky tests. The CI run looks good.

@kunalspathak
Contributor

The CI run looks good.

Can someone sign off on this PR then, so we can go ahead and move the repo?

// cc: @thefourtheye, @silverwind, @jepler, @Fishrock123, @benjamingr , @TooTallNate, @jasnell

@rvagg
Member
rvagg commented Feb 10, 2016

@orangemocha I think we're good to go and you should have the right access level to pull it off.

I'd like to suggest a couple of things:

  1. Keep it in sync with nodejs/node as much as possible so it can be compared at any time by anyone who adds it as a remote to their local clone. Either keep this PR open with the changes (which I think you were suggesting) or keep an equivalent PR open there with the changes. The aim is to get it to a point where we can actually make a decision about merging it or not rather than pursuing it as a separate project indefinitely (at least that's the aim right now).
  2. Place a notice at the top of the README on the default branch of the new repo that clearly states that it is a work in progress, not an officially endorsed branch of Node.js and maybe link back to this issue. We need to follow a process that doesn't presume that decisions have been made that have actually not, so far all that the CTC has offered is a way to make progress and I imagine we'd get a lot of upset people commenting if it looked like we'd already pulled the trigger on this.
@orangemocha
Member

Thanks for the suggestions, @rvagg.

  1. Yep, I had a chat with @kunalspathak and the current plan is to have a master branch which is the same as nodejs/node:master and a chakracore-master branch that would be master + the chakracore changes, so the diff would be easily visible, and to keep syncing master from nodejs/node and integrating changes into chakracore-master, ~weekly. Let me know if you have any additional input.
  2. Will do.
@silverwind
Contributor

Sorry to bikeshed, but I'm still not totally happy with the name of process.jsEngine. I'd rather have it be process.engine for simplicity and it's more neutral towards the es vs js naming debacle.

@isiahmeadows

@silverwind 👍, but more so for API consistency than that particular reason. Most of the other Node APIs use that convention. It's version, not nodeVersion, for example. Trying not to derail the topic, though.

@orangemocha
Member

I am on the opposite camp, I think that engine could be ambiguous. There could be different types of engines in this thing (e.g. libuv could be seen as the I/O engine).

Anyway, since the new repo is already live, at this point I think it would be better to open an issue in the new repo and discuss it there. I will follow up.

EDIT: opened issue nodejs/node-chakracore#19

@orangemocha
Member

And here is node-chakracore: https://github.com/nodejs/node-chakracore ! Hoping to get lots of contributions 😄

@MylesBorins MylesBorins added a commit that referenced this pull request Feb 17, 2016
@vkurchatkin @MylesBorins vkurchatkin + MylesBorins test: remove Object.observe from tests
Testing this wasn't really useful, besides Object.observe is going to be
deprecated.

Also this test fails with Chakra (#4765) for obvious reason.

PR-URL: #4769
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
78fc5e3
@MylesBorins MylesBorins added a commit that referenced this pull request Feb 18, 2016
@vkurchatkin @MylesBorins vkurchatkin + MylesBorins test: remove Object.observe from tests
Testing this wasn't really useful, besides Object.observe is going to be
deprecated.

Also this test fails with Chakra (#4765) for obvious reason.

PR-URL: #4769
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
9e6f363
@joaocgreis joaocgreis referenced this pull request in nodejs/help Feb 19, 2016
Closed

Does NodeJS run in Windows CE? #93

@kunalspathak
Contributor

As @orangemocha mentioned these changes are ported to new repo https://github.com/nodejs/node-chakracore chakracore-master branch. Weekly merge from nodejs.node master and chakrashim changes will happen in this branch of new repo. Feel free to contribute and providing feedback. Thanks everyone for the support!

@ChALkeR
Member
ChALkeR commented Feb 19, 2016

@orangemocha Does this still need ctc-agenda?

@orangemocha orangemocha removed the ctc-agenda label Feb 19, 2016
@orangemocha
Member

Removed ctc-agenda. We'll ping the CTC periodically to provide updates and get feedback on the direction of the project. Thank you.

@MylesBorins MylesBorins added a commit that referenced this pull request Mar 2, 2016
@vkurchatkin @MylesBorins vkurchatkin + MylesBorins test: remove Object.observe from tests
Testing this wasn't really useful, besides Object.observe is going to be
deprecated.

Also this test fails with Chakra (#4765) for obvious reason.

PR-URL: #4769
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
c490464
@itsjavi
itsjavi commented Mar 23, 2016

Abstracting Node from the underlying engine sounds good, but we should avoid incompatibilities, otherwise we may start to see packages only working for a specific engine that will not be reusable in the other, so here comes the potential fragmentation.
Please, let's not bring cross-platform headache to Node, make the differences minimal, defining a clear and common API, that's all I would ask.
Great job btw.

@isiahmeadows

@mjolnic I agree, but I will note that Browserify/Webpack isn't helping that too much. I do feel that NAN might help alleviate much of that concern, though, but it'll have to go through a major version change to account for all the additional abstraction boilerplate it'll need.

I'm also staring intently on WebAssembly, which will also be runnable on future versions of Node, with Chakra and V8 both supporting it, and that may have an even more significant impact on native addons (more than even supporting Chakra IMHO, because of the language difference).

@isiahmeadows isiahmeadows referenced this pull request in WebAssembly/design Mar 26, 2016
Open

JS interop and glue code #119

@jfbastien

From Node's POV WebAssembly will look like an opaque blob of JS. You can call into it from JS, it can call out to JS, but the API surface is the same as an opaque JS blob.

Eventually WebAssembly may gain access to extra APIs, but the MVP won't have anything special. If / when WebAssembly gains new APIs I think they should be designed with Node in mind, not just browsers.

@scovetta scovetta pushed a commit to scovetta/node that referenced this pull request Apr 2, 2016
@vkurchatkin vkurchatkin + Michael Scovetta test: remove Object.observe from tests
Testing this wasn't really useful, besides Object.observe is going to be
deprecated.

Also this test fails with Chakra (#4765) for obvious reason.

PR-URL: nodejs#4769
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
7225c8c
@kunalspathak kunalspathak added a commit to kunalspathak/node-gyp that referenced this pull request Dec 12, 2016
@kunalspathak kunalspathak gyp: Add support to build node.js with chakracore
Microsoft's chakracore engine is dependent on Windows SDK, and build tools
should know the version installed on user machine. This change adds those
dependencies in node-gyp tools. Below is the summary:

* Configure msvs_windows_target_platform_version to use the right
   Windows SDK.
* Configure msvs_use_library_dependency_inputs to export symbols
   correctly (otherwise functions not used by node.exe but might be
   needed by native addon modules could be optimized away by linker).

 These changes were originally made in nodejs/node#4765, but as @shigeki
 mentioned, it was more sensible to send these changes as PR to node-gyp
 repo.
79f1c5a
@kunalspathak kunalspathak added a commit to kunalspathak/node-gyp that referenced this pull request Jan 19, 2017
@kunalspathak kunalspathak gyp: Add support to build node.js with chakracore
Microsoft's chakracore engine is dependent on Windows SDK, and build tools
should know the version installed on user machine. This change adds those
dependencies in node-gyp tools. Below is the summary:

* Configure msvs_windows_target_platform_version to use the right
   Windows SDK.
* Configure msvs_use_library_dependency_inputs to export symbols
   correctly (otherwise functions not used by node.exe but might be
   needed by native addon modules could be optimized away by linker).

 These changes were originally made in nodejs/node#4765, but as @shigeki
 mentioned, it was more sensible to send these changes as PR to node-gyp
 repo.
7e56293
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment