Changed templating-tools to require uglify-js from NPM #234

Merged
merged 19 commits into from Feb 25, 2017

Conversation

Projects
None yet
4 participants
@sethmurphy18
Contributor

sethmurphy18 commented Feb 22, 2017

I have been working with @abernix and @benjamn to replace uglify with the babili minifer in the minifier-js package to provide better ES6 support in Meteor (see meteor/meteor#8378 and meteor/meteor#8397). We have hit a roadblock, though, because the templating-tools tests are failing. I have made the changes suggested by @abernix that will make the code pass. The minifier is actually removing some of the newlines in the code, causing the tests to fail. This PR would fix that issue. I also had to update the version of node that Circle is using because Babili requires at least Node 4.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 22, 2017

Contributor

Ok, so over here I get the result that I should be getting in the Meteor repo. I think I know why though, for your tests you are pulling the package from Atmosphere, whereas over there it comes from the repo. Just a theory here.

Contributor

sethmurphy18 commented Feb 22, 2017

Ok, so over here I get the result that I should be getting in the Meteor repo. I think I know why though, for your tests you are pulling the package from Atmosphere, whereas over there it comes from the repo. Just a theory here.

@mitar

This comment has been minimized.

Show comment
Hide comment
@mitar

mitar Feb 22, 2017

Collaborator

I am unsure why would system's node be important here? To my understanding Meteor should bundle NPM with it and have it used it to install Babili?

Collaborator

mitar commented Feb 22, 2017

I am unsure why would system's node be important here? To my understanding Meteor should bundle NPM with it and have it used it to install Babili?

@benjamn

This comment has been minimized.

Show comment
Hide comment
@benjamn

benjamn Feb 22, 2017

Member

@mitar The Babili minifier won't run in Node versions older than 4.

Member

benjamn commented Feb 22, 2017

@mitar The Babili minifier won't run in Node versions older than 4.

@mitar

This comment has been minimized.

Show comment
Hide comment
@mitar

mitar Feb 22, 2017

Collaborator

But why would system's node be used? I thought Meteor is not using it anymore?

Collaborator

mitar commented Feb 22, 2017

But why would system's node be used? I thought Meteor is not using it anymore?

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

@benjamn, I think @mitar is right here, Meteor doesn't use the system's version of node usually, in the Meteor repo itself, we have to use the correct version because we are building Meteor, but over here they aren't. They just download a release and use the devkit like most other people would. I will change that back.

Contributor

sethmurphy18 commented Feb 23, 2017

@benjamn, I think @mitar is right here, Meteor doesn't use the system's version of node usually, in the Meteor repo itself, we have to use the correct version because we are building Meteor, but over here they aren't. They just download a release and use the devkit like most other people would. I will change that back.

@abernix

This comment has been minimized.

Show comment
Hide comment
@abernix

abernix Feb 23, 2017

Member

I agree that changing the circle.yml Node version here is unnecessary and that can be changed back.

The Node version here is only used for the Hexo docs if I'm not mistaken. With that being said, I don't really see a harm in bumping the Node version for that to a more recent version. Regardless, that's a task for a separate PR/commit.

Member

abernix commented Feb 23, 2017

I agree that changing the circle.yml Node version here is unnecessary and that can be changed back.

The Node version here is only used for the Hexo docs if I'm not mistaken. With that being said, I don't really see a harm in bumping the Node version for that to a more recent version. Regardless, that's a task for a separate PR/commit.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

On a side note, that one test will continue to fail over here since Blaze does not use the meteor/devel version of minifier-js, but that change will make the meteor test pass so we can merge the change over there. I would update the dependency, but I am not sure if Meteor will pull the one from github or not.

Contributor

sethmurphy18 commented Feb 23, 2017

On a side note, that one test will continue to fail over here since Blaze does not use the meteor/devel version of minifier-js, but that change will make the meteor test pass so we can merge the change over there. I would update the dependency, but I am not sure if Meteor will pull the one from github or not.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

I think I see where @mitar was coming from in asking. If something isn't broke it doesn't need fixing, and from what I can see, he is one of the main people over here in Blaze, so he knows better than anyone what changes need made.

Contributor

sethmurphy18 commented Feb 23, 2017

I think I see where @mitar was coming from in asking. If something isn't broke it doesn't need fixing, and from what I can see, he is one of the main people over here in Blaze, so he knows better than anyone what changes need made.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

I found our problem. Digging further into the Blaze code reveals that there is now a complete incompatibility between Blaze and Meteor because of this change. Blaze is using the UglifierJS beautify function, which Babili does not support, Babili is one way. So, instead of making their test pass, I think we should just have Blaze use uglify on their end as their own package.

Contributor

sethmurphy18 commented Feb 23, 2017

I found our problem. Digging further into the Blaze code reveals that there is now a complete incompatibility between Blaze and Meteor because of this change. Blaze is using the UglifierJS beautify function, which Babili does not support, Babili is one way. So, instead of making their test pass, I think we should just have Blaze use uglify on their end as their own package.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

That said, I can do one of two things for you guys, I can rewrite spacebars-compiler to use UglifyJS from npm, or I can create a new package exclusively for Blaze that exports UglifyJS, up to you @mitar, this is more your project.

Contributor

sethmurphy18 commented Feb 23, 2017

That said, I can do one of two things for you guys, I can rewrite spacebars-compiler to use UglifyJS from npm, or I can create a new package exclusively for Blaze that exports UglifyJS, up to you @mitar, this is more your project.

@abernix

This comment has been minimized.

Show comment
Hide comment
@abernix

abernix Feb 23, 2017

Member

We could consider just pinning minifier-js to the current version which uses UglifyJS in spacebars-compilers package.js:

  api.use('minifier-js@=1.2.17', ['server'], { weak: true });
Member

abernix commented Feb 23, 2017

We could consider just pinning minifier-js to the current version which uses UglifyJS in spacebars-compilers package.js:

  api.use('minifier-js@=1.2.17', ['server'], { weak: true });
@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

That is a much better option than I came up with, I will update it and push it, I hate to have so many pushes, but my computer is sooo slow that tests take hours to run, even a single package. Gotta love rocking a single core AMD.

Contributor

sethmurphy18 commented Feb 23, 2017

That is a much better option than I came up with, I will update it and push it, I hate to have so many pushes, but my computer is sooo slow that tests take hours to run, even a single package. Gotta love rocking a single core AMD.

@abernix

This comment has been minimized.

Show comment
Hide comment
@abernix

abernix Feb 23, 2017

Member

Not a huge deal, I don't think. Don't be afraid to push new commits either (instead of amending previous commits). It might make it a lot more clear how things have progressed. We can always squash at the end!

Member

abernix commented Feb 23, 2017

Not a huge deal, I don't think. Don't be afraid to push new commits either (instead of amending previous commits). It might make it a lot more clear how things have progressed. We can always squash at the end!

@sethmurphy18 sethmurphy18 changed the title from Updated html-scanner-tests to work with new minifier to Changed minifier-js to stay at 1.2.17 instead of Meteor's 2.0.0 Feb 23, 2017

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

I will make a push on my test branch of meteor and run my Travis build to see if it passes now, so I am not using MDG Travis resources.

Contributor

sethmurphy18 commented Feb 23, 2017

I will make a push on my test branch of meteor and run my Travis build to see if it passes now, so I am not using MDG Travis resources.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Actually, I have just been rebasing and squashing as I go, to reduce the workload on you guys.

Contributor

sethmurphy18 commented Feb 23, 2017

Actually, I have just been rebasing and squashing as I go, to reduce the workload on you guys.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Alright! Blaze now passes, even though it fails because of my lack of the S3 keys to push the docs...you might wanna update that in the CI script, since passing CI is required to merge.

Contributor

sethmurphy18 commented Feb 23, 2017

Alright! Blaze now passes, even though it fails because of my lack of the S3 keys to push the docs...you might wanna update that in the CI script, since passing CI is required to merge.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Ok, don't merge, meteor still won't pass with this change. Something is not right here, and I cannot peg it. If I run the test from the Blaze repo it passes, but not Meteor, I wonder if the problem is related to not having two versions of the same package installed at once. I need to modify the Travis script right quick to print the versions file to the console, and I will confirm whether or not that is the case.

Contributor

sethmurphy18 commented Feb 23, 2017

Ok, don't merge, meteor still won't pass with this change. Something is not right here, and I cannot peg it. If I run the test from the Blaze repo it passes, but not Meteor, I wonder if the problem is related to not having two versions of the same package installed at once. I need to modify the Travis script right quick to print the versions file to the console, and I will confirm whether or not that is the case.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Found the issue, you can only have 1 version of a package installed at a time, and the version of minifier-js that is installed is the new version, which is okay for the spacebars compiler because it has a failsafe in case it isn't there, but I am going to change the test right quick and you will understand everything, this will make both meteor and blaze pass all their tests.

Contributor

sethmurphy18 commented Feb 23, 2017

Found the issue, you can only have 1 version of a package installed at a time, and the version of minifier-js that is installed is the new version, which is okay for the spacebars compiler because it has a failsafe in case it isn't there, but I am going to change the test right quick and you will understand everything, this will make both meteor and blaze pass all their tests.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Nope, still failing on meteor, I think the issue does go back to not having beautify available. I am going to try reworking the Blaze package to bring in uglify and see if that helps. Since Blaze isn't really working with ES6, they should be okay with uglify, and the code is going to be reminified with Meteor anyway later.

Contributor

sethmurphy18 commented Feb 23, 2017

Nope, still failing on meteor, I think the issue does go back to not having beautify available. I am going to try reworking the Blaze package to bring in uglify and see if that helps. Since Blaze isn't really working with ES6, they should be okay with uglify, and the code is going to be reminified with Meteor anyway later.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

For those who are interested, the issue that is causing the test to fail is here.

Contributor

sethmurphy18 commented Feb 23, 2017

For those who are interested, the issue that is causing the test to fail is here.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

So, now we have to figure out how to cope with this. minifer-js is no longer a weak dependency of spacebars-compiler, it now ships with uglify-js, if this update goes through. I am trying to think of alternate solutions, but for testing, I am seeing if this will make Blaze and Meteor pass all their tests.

Contributor

sethmurphy18 commented Feb 23, 2017

So, now we have to figure out how to cope with this. minifer-js is no longer a weak dependency of spacebars-compiler, it now ships with uglify-js, if this update goes through. I am trying to think of alternate solutions, but for testing, I am seeing if this will make Blaze and Meteor pass all their tests.

Seth Murphy
@abernix

This comment has been minimized.

Show comment
Hide comment
@abernix

abernix Feb 23, 2017

Member

I have just been rebasing and squashing as I go, to reduce the workload on you guys.

I meant you don't need to do that, and it might be helpful if you didn't. Squashing is one button at the end and it's actually nice to see what you've changed/tried in each commit to provide advice instead of trying to guess.

I am seeing if this will make Blaze and Meteor pass all their tests.

I think you actually have a circular dependency that may not be getting considered here and I don't know if you can actually fix the tests without . meteors repo has a submodule to the blaze repository which places it at packages/non-core/blaze. Typically, that repo points to the latest stable release of Blaze.

If you haven't changed that submodule reference to point to the modified Blaze branch with the fixed html-scanner-tests, you will have failures on the meteor branch.

I think the beautify issue is certainly something that needed to be addressed, but presumably the version pinning fixed it?

Member

abernix commented Feb 23, 2017

I have just been rebasing and squashing as I go, to reduce the workload on you guys.

I meant you don't need to do that, and it might be helpful if you didn't. Squashing is one button at the end and it's actually nice to see what you've changed/tried in each commit to provide advice instead of trying to guess.

I am seeing if this will make Blaze and Meteor pass all their tests.

I think you actually have a circular dependency that may not be getting considered here and I don't know if you can actually fix the tests without . meteors repo has a submodule to the blaze repository which places it at packages/non-core/blaze. Typically, that repo points to the latest stable release of Blaze.

If you haven't changed that submodule reference to point to the modified Blaze branch with the fixed html-scanner-tests, you will have failures on the meteor branch.

I think the beautify issue is certainly something that needed to be addressed, but presumably the version pinning fixed it?

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Ah okay, I will stop squashing my commits then. The thing is, I see why you say circular dependency, because spacebars-compiler is not seeing the version of minify-js that has the UglifyJS, but templating-tools is, and I think it is because for spacebars-compiler it is a weak dependency, whereas with templating-tools it is a strong dependency. As for the submodule, I updated it to use my git repo, while I am testing. I am honestly now sure why the output needs to be beautified anyway, but I don't really want to make a major change like that without permission.

Contributor

sethmurphy18 commented Feb 23, 2017

Ah okay, I will stop squashing my commits then. The thing is, I see why you say circular dependency, because spacebars-compiler is not seeing the version of minify-js that has the UglifyJS, but templating-tools is, and I think it is because for spacebars-compiler it is a weak dependency, whereas with templating-tools it is a strong dependency. As for the submodule, I updated it to use my git repo, while I am testing. I am honestly now sure why the output needs to be beautified anyway, but I don't really want to make a major change like that without permission.

Seth Murphy
@abernix

This comment has been minimized.

Show comment
Hide comment
@abernix

abernix Feb 23, 2017

Member

I don't think you should Npm.depends on uglify-js directly in this package. I think that would be defeating the exact thing the weak dependency is accomplishing.

You bring up a great point about "beautify" though. @mitar, do you understand why we need beautify?

It honestly sounds like things are actually good at the moment, even if the tests seem to indicate otherwise. I think if you run the CircleCI tests against this branch, you will run into problems because minifier-js is going to grab the published version. You might be able to work around this temporarily by cloning the fixed minifier-js into the packages folder here in Blaze.

Then, for the Meteor tests to pass, you would need to point the submodule at the same repo and branch as the fixed Blaze problem (the one you would have tested above) by changing the submodule itself (probably by editing .gitmodules in the root of meteor and then running git submodule sync?).

I'm actually terribly sorry that this is so difficult on the testing front, but this is a dependency chain that we haven't really had to test since before the repositories were split. Frankly, just releasing it would work, but getting the tests to play nicely is the more significant problem here. It would be certainly be easier if you could run the tests locally on your computer, but no big deal.

Also, the AWS errors will persist since you don't have AWS credentials to publish the docs from your branch. I suppose we can probably fix that, but we can review the test results ourselves and force-merge if necessary.

All in all, I think you can probably put this down at the moment, at least until we get around to doing some testing and review ourselves which might be a few days.

Member

abernix commented Feb 23, 2017

I don't think you should Npm.depends on uglify-js directly in this package. I think that would be defeating the exact thing the weak dependency is accomplishing.

You bring up a great point about "beautify" though. @mitar, do you understand why we need beautify?

It honestly sounds like things are actually good at the moment, even if the tests seem to indicate otherwise. I think if you run the CircleCI tests against this branch, you will run into problems because minifier-js is going to grab the published version. You might be able to work around this temporarily by cloning the fixed minifier-js into the packages folder here in Blaze.

Then, for the Meteor tests to pass, you would need to point the submodule at the same repo and branch as the fixed Blaze problem (the one you would have tested above) by changing the submodule itself (probably by editing .gitmodules in the root of meteor and then running git submodule sync?).

I'm actually terribly sorry that this is so difficult on the testing front, but this is a dependency chain that we haven't really had to test since before the repositories were split. Frankly, just releasing it would work, but getting the tests to play nicely is the more significant problem here. It would be certainly be easier if you could run the tests locally on your computer, but no big deal.

Also, the AWS errors will persist since you don't have AWS credentials to publish the docs from your branch. I suppose we can probably fix that, but we can review the test results ourselves and force-merge if necessary.

All in all, I think you can probably put this down at the moment, at least until we get around to doing some testing and review ourselves which might be a few days.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Yeah, all I did was update my .gitmodules file to point to my repo for packages/non-core/blaze then pushed and Travis clones my repo instead of Blaze's, I won't push that onto the actual git repo, it's on my testing branch. Honestly, even while requiring uglify-js as a hard-coded npm dependency, it still fails because the code is not getting beautified on the meteor side. This is my search results in the entire code for the SpacebarCompiler._beautify function, so to me, it seems to serve little if any purpose:

Searching 119 files for "SpacebarsCompiler._beautify"

C:\Users\smurp\Projects\blaze\packages\spacebars-compiler\compile_tests.js:
   36          postProcess(output.toString()),
   37          postProcess(
   38:           SpacebarsCompiler._beautify('(' + expected.toString() + ')')),
   39          input);
   40      }

C:\Users\smurp\Projects\blaze\packages\spacebars-compiler\compiler.js:
   90    code += '; })';
   91  
   92:   code = SpacebarsCompiler._beautify(code);
   93  
   94    return code;
   95  };
   96  
   97: SpacebarsCompiler._beautify = function (code) {
   98    var uglify = Npm.require('uglify-js');
   99    var result = uglify.minify(

3 matches across 2 files
Contributor

sethmurphy18 commented Feb 23, 2017

Yeah, all I did was update my .gitmodules file to point to my repo for packages/non-core/blaze then pushed and Travis clones my repo instead of Blaze's, I won't push that onto the actual git repo, it's on my testing branch. Honestly, even while requiring uglify-js as a hard-coded npm dependency, it still fails because the code is not getting beautified on the meteor side. This is my search results in the entire code for the SpacebarCompiler._beautify function, so to me, it seems to serve little if any purpose:

Searching 119 files for "SpacebarsCompiler._beautify"

C:\Users\smurp\Projects\blaze\packages\spacebars-compiler\compile_tests.js:
   36          postProcess(output.toString()),
   37          postProcess(
   38:           SpacebarsCompiler._beautify('(' + expected.toString() + ')')),
   39          input);
   40      }

C:\Users\smurp\Projects\blaze\packages\spacebars-compiler\compiler.js:
   90    code += '; })';
   91  
   92:   code = SpacebarsCompiler._beautify(code);
   93  
   94    return code;
   95  };
   96  
   97: SpacebarsCompiler._beautify = function (code) {
   98    var uglify = Npm.require('uglify-js');
   99    var result = uglify.minify(

3 matches across 2 files
@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

I will try removing that code and see what happens, if @mitar isn't happy with the result, he can stop the merge.

Contributor

sethmurphy18 commented Feb 23, 2017

I will try removing that code and see what happens, if @mitar isn't happy with the result, he can stop the merge.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

OK, I am going to wait on input from the Blaze team, this is pretty deeply intertwined into their tests.

Contributor

sethmurphy18 commented Feb 23, 2017

OK, I am going to wait on input from the Blaze team, this is pretty deeply intertwined into their tests.

@mitar

This comment has been minimized.

Show comment
Hide comment
@mitar

mitar Feb 23, 2017

Collaborator

You bring up a great point about "beautify" though. @mitar, do you understand why we need beautify?

I am not really sure. My guess is that it wants to make code look nice before minimization. So Spacebars JS code gets automatically generated for each template. And that generated sting is really ugly (but not minified). So make it nicely formatted helps during non-production development.

I think the best way here would be to simply use some other package to do that. @sethmurphy18, do you know of any other library doing that (not sure if we need ES6 support here, I would guess no)? Alternatively, we could just directly depend on uglify through Npm.depends in spacebars compiler package and this is it. Because it is a server-side only thing, I do not think we have to worry about having extra dependencies there.

Collaborator

mitar commented Feb 23, 2017

You bring up a great point about "beautify" though. @mitar, do you understand why we need beautify?

I am not really sure. My guess is that it wants to make code look nice before minimization. So Spacebars JS code gets automatically generated for each template. And that generated sting is really ugly (but not minified). So make it nicely formatted helps during non-production development.

I think the best way here would be to simply use some other package to do that. @sethmurphy18, do you know of any other library doing that (not sure if we need ES6 support here, I would guess no)? Alternatively, we could just directly depend on uglify through Npm.depends in spacebars compiler package and this is it. Because it is a server-side only thing, I do not think we have to worry about having extra dependencies there.

packages/templating-tools/package.js
- // boilerplate-generator uses spacebars-compiler.)
- // XXX maybe uglify should be applied by this plugin instead of via magic
- // weak dependency.
- 'minifier-js@1.2.14'

This comment has been minimized.

@mitar

mitar Feb 23, 2017

Collaborator

So I think we still want minimization to happen and that can be a normal dependency on a new minifier.

@mitar

mitar Feb 23, 2017

Collaborator

So I think we still want minimization to happen and that can be a normal dependency on a new minifier.

- // The templating plugin will pull in minifier-js, so that generated code will
- // be beautified. But it's a weak dependency so that eg boilerplate-generator
- // doesn't pull in the minifier.
- api.use('minifier-js@1.2.14', ['server'], { weak: true });

This comment has been minimized.

@mitar

mitar Feb 23, 2017

Collaborator

But we can get rid of this one and just have a Npm.depends on the NPM module.

@mitar

mitar Feb 23, 2017

Collaborator

But we can get rid of this one and just have a Npm.depends on the NPM module.

packages/spacebars-compiler/compiler.js
- // don't actually beautify; no UglifyJS
- return code;
- }
+ var uglify = Npm.require('uglify-js');

This comment has been minimized.

@mitar

mitar Feb 23, 2017

Collaborator

I would move this line to the top of the file, but otherwise yes, something like this looks good.

@mitar

mitar Feb 23, 2017

Collaborator

I would move this line to the top of the file, but otherwise yes, something like this looks good.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

I'll look into the changes you made, and see if meteor will pass that way. Also, the only other library I can find for doing this is beautify-js, which may work better since it doesn't have as many dependencies.

Contributor

sethmurphy18 commented Feb 23, 2017

I'll look into the changes you made, and see if meteor will pass that way. Also, the only other library I can find for doing this is beautify-js, which may work better since it doesn't have as many dependencies.

Seth Murphy
@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Still failing on the meteor side, even with that change.

Contributor

sethmurphy18 commented Feb 23, 2017

Still failing on the meteor side, even with that change.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Also from the Meteor Guide on writing packages:

One important feature of the Atmosphere package system is that it is single-loading: no two packages in the same app can have dependencies on conflicting versions of a single package. Read more about that in the section about version constraints below.
Contributor

sethmurphy18 commented Feb 23, 2017

Also from the Meteor Guide on writing packages:

One important feature of the Atmosphere package system is that it is single-loading: no two packages in the same app can have dependencies on conflicting versions of a single package. Read more about that in the section about version constraints below.
Seth Murphy
@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Also, a question. Is the npm-shrinkwrap.json even required anymore? The docs on creating packages don't mention it. Is Npm.depends() sufficient now?

Contributor

sethmurphy18 commented Feb 23, 2017

Also, a question. Is the npm-shrinkwrap.json even required anymore? The docs on creating packages don't mention it. Is Npm.depends() sufficient now?

Seth Murphy added some commits Feb 23, 2017

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 23, 2017

Contributor

Ok, so I keep getting this error and I cannot figure out why, I get the error if the folder is there, and I get it if it's not there. If I remove the .npm folder entirely, then meteor --get-ready just crashes.

spacebars-compiler: updating npm dependencies -- js-beautify...
/tmp/meteor/.meteor/packages/coffeescript/.1.11.1_4.1rnji0d++os+web.browser+web.cordova/plugin.compileCoffeescript.os/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:190
      throw error;
      ^

Error: ENOENT: no such file or directory, lstat '/home/ubuntu/blaze/packages/spacebars-compiler/.npm/package/node_modules'
Contributor

sethmurphy18 commented Feb 23, 2017

Ok, so I keep getting this error and I cannot figure out why, I get the error if the folder is there, and I get it if it's not there. If I remove the .npm folder entirely, then meteor --get-ready just crashes.

spacebars-compiler: updating npm dependencies -- js-beautify...
/tmp/meteor/.meteor/packages/coffeescript/.1.11.1_4.1rnji0d++os+web.browser+web.cordova/plugin.compileCoffeescript.os/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_server.js:190
      throw error;
      ^

Error: ENOENT: no such file or directory, lstat '/home/ubuntu/blaze/packages/spacebars-compiler/.npm/package/node_modules'
Seth Murphy
@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 24, 2017

Contributor

Ok, given that I am having these issues with this. I have an idea, and I want MDG guidance. uglify-js is smaller than js-beautify by almost 100%. I can do 1 of 2 things in my mind, because this simply will not work. I can 1) create my own atmosphere package that does this, or 2) create a new package in Meteor Core that exports a beautify function. both are going to add some bloat to this package, ~1-2 MB, but if Blaze absolutely wants to keep the beautification, this is essential. For some reason, NPM packages won't install directly into here. @mitar your input is essential here, since you seem to be for the most part running Blaze.

Contributor

sethmurphy18 commented Feb 24, 2017

Ok, given that I am having these issues with this. I have an idea, and I want MDG guidance. uglify-js is smaller than js-beautify by almost 100%. I can do 1 of 2 things in my mind, because this simply will not work. I can 1) create my own atmosphere package that does this, or 2) create a new package in Meteor Core that exports a beautify function. both are going to add some bloat to this package, ~1-2 MB, but if Blaze absolutely wants to keep the beautification, this is essential. For some reason, NPM packages won't install directly into here. @mitar your input is essential here, since you seem to be for the most part running Blaze.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Ok, so I found out why npm packages don't install over here. It turns out Blaze never goes through the build process like you guys do in the meteor repo. So, that is why it never finds the node_modules folder. It actually does find it, but it doesn't find its contents because they are never installed.

Contributor

sethmurphy18 commented Feb 25, 2017

Ok, so I found out why npm packages don't install over here. It turns out Blaze never goes through the build process like you guys do in the meteor repo. So, that is why it never finds the node_modules folder. It actually does find it, but it doesn't find its contents because they are never installed.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Nope wrong yet again. Turns out meteor is installing the npm dependencies, it just takes too long to complete. The timeout is 300ms, which is not long enough for meteor to install the npm packages. It might be best if we simply create a new package in meteor core for this. @abernix do you have any input on the issue?

Contributor

sethmurphy18 commented Feb 25, 2017

Nope wrong yet again. Turns out meteor is installing the npm dependencies, it just takes too long to complete. The timeout is 300ms, which is not long enough for meteor to install the npm packages. It might be best if we simply create a new package in meteor core for this. @abernix do you have any input on the issue?

@mitar

This comment has been minimized.

Show comment
Hide comment
@mitar

mitar Feb 25, 2017

Collaborator

So I think we should depend on the NPM package. I do not know why this does not work.

Collaborator

mitar commented Feb 25, 2017

So I think we should depend on the NPM package. I do not know why this does not work.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

@mitar It's the timeout, installing the npm packages causees the build to take more than 300 seconds. I'm testing a build config with a timeout of 30,000 seconds, also using my version of meteor, so I can remove as many variables as possible. I am still having an issue with meteor --get-ready dying unexpectedly. So I need to check the meteor logs to see what is causing that.

Contributor

sethmurphy18 commented Feb 25, 2017

@mitar It's the timeout, installing the npm packages causees the build to take more than 300 seconds. I'm testing a build config with a timeout of 30,000 seconds, also using my version of meteor, so I can remove as many variables as possible. I am still having an issue with meteor --get-ready dying unexpectedly. So I need to check the meteor logs to see what is causing that.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Or you know, I could just still be depending on a package that I didn't include in the shrinkwrap. That might just be it.

Contributor

sethmurphy18 commented Feb 25, 2017

Or you know, I could just still be depending on a package that I didn't include in the shrinkwrap. That might just be it.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Also, in my circle.yml, I updated circle to only try to deploy the docs on tags, I think that might be a good option, or just simply having a different branch for your docs, that way circle tests will actually pass whenever someone tries to pull and you don't have to check circle manually to see if it was the docs that failed or something else.

Contributor

sethmurphy18 commented Feb 25, 2017

Also, in my circle.yml, I updated circle to only try to deploy the docs on tags, I think that might be a good option, or just simply having a different branch for your docs, that way circle tests will actually pass whenever someone tries to pull and you don't have to check circle manually to see if it was the docs that failed or something else.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Ok, so I ran an ssh on one of my builds and I am completely perplexed. The node_modules directory is there, so we can rule out the timeout issue. It seems like coffeescript is trying to run before the node_modules directory gets created. I am going to try to convert the coffee tests to pure javascript, commit that on master, and see what happens.

Contributor

sethmurphy18 commented Feb 25, 2017

Ok, so I ran an ssh on one of my builds and I am completely perplexed. The node_modules directory is there, so we can rule out the timeout issue. It seems like coffeescript is trying to run before the node_modules directory gets created. I am going to try to convert the coffee tests to pure javascript, commit that on master, and see what happens.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Ok, I have the problem pegged now, and I need MDG help to fix it. I have been testing the package locally, and what happens is while building the package, meteor actually installs uglify-js, then when it builds the web.browser target it deletes everything in the .npm folder, and I am not sure why. Do you have any suggestions @abernix ?

EDIT: Nevermind, I fixed it, turns out I need to Npm.depends() in Package.onTest() as well.

Contributor

sethmurphy18 commented Feb 25, 2017

Ok, I have the problem pegged now, and I need MDG help to fix it. I have been testing the package locally, and what happens is while building the package, meteor actually installs uglify-js, then when it builds the web.browser target it deletes everything in the .npm folder, and I am not sure why. Do you have any suggestions @abernix ?

EDIT: Nevermind, I fixed it, turns out I need to Npm.depends() in Package.onTest() as well.

Seth Murphy

@sethmurphy18 sethmurphy18 changed the title from Changed minifier-js to stay at 1.2.17 instead of Meteor's 2.0.0 to Changed templating-tools to require uglify-js from NPM Feb 25, 2017

Seth Murphy added some commits Feb 25, 2017

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Great news @mitar. I am done and will be out of your hair. All of the code is building correctly and all tests pass. You can merge whenever you want. Then we can update the reference in the meteor repo and meteor should build and pass all of its tests and we can close these PRs.

Contributor

sethmurphy18 commented Feb 25, 2017

Great news @mitar. I am done and will be out of your hair. All of the code is building correctly and all tests pass. You can merge whenever you want. Then we can update the reference in the meteor repo and meteor should build and pass all of its tests and we can close these PRs.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 25, 2017

Contributor

Ok, not done yet, now the blaze tests are failing. In particular video:


C: tinytest - blaze - render - video - autoplay : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"autoplay","events":[{"sequence":10139,"type":"fail","details":{"type":"assert_equal","expected":"false","actual":"true","not":false},"cookie":{"name":"blaze - render - video - autoplay","offset":1,"groupPath":["tinytest","blaze","render","video"],"shortName":"autoplay"}}]}
C: tinytest - blaze - render - video - controls : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"controls","events":[{"sequence":10142,"type":"fail","details":{"type":"assert_equal","expected":"false","actual":"true","not":false},"cookie":{"name":"blaze - render - video - controls","offset":1,"groupPath":["tinytest","blaze","render","video"],"shortName":"controls"}}]}
C: tinytest - blaze - render - video - loop : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"loop","events":[{"sequence":10145,"type":"fail","details":{"type":"assert_equal","expected":"false","actual":"true","not":false},"cookie":{"name":"blaze - render - video - loop","offset":1,"groupPath":["tinytest","blaze","render","video"],"shortName":"loop"}}]}
C: tinytest - blaze - render - video - muted : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"muted","events":[{"sequence":10147,"type":"fail","details":{"type":"assert_equal","expected":"false","not":false},"cookie":{"name":"blaze - render - video - muted","offset":0,"groupPath":["tinytest","blaze","render","video"],"shortName":"muted"}}]}
Contributor

sethmurphy18 commented Feb 25, 2017

Ok, not done yet, now the blaze tests are failing. In particular video:


C: tinytest - blaze - render - video - autoplay : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"autoplay","events":[{"sequence":10139,"type":"fail","details":{"type":"assert_equal","expected":"false","actual":"true","not":false},"cookie":{"name":"blaze - render - video - autoplay","offset":1,"groupPath":["tinytest","blaze","render","video"],"shortName":"autoplay"}}]}
C: tinytest - blaze - render - video - controls : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"controls","events":[{"sequence":10142,"type":"fail","details":{"type":"assert_equal","expected":"false","actual":"true","not":false},"cookie":{"name":"blaze - render - video - controls","offset":1,"groupPath":["tinytest","blaze","render","video"],"shortName":"controls"}}]}
C: tinytest - blaze - render - video - loop : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"loop","events":[{"sequence":10145,"type":"fail","details":{"type":"assert_equal","expected":"false","actual":"true","not":false},"cookie":{"name":"blaze - render - video - loop","offset":1,"groupPath":["tinytest","blaze","render","video"],"shortName":"loop"}}]}
C: tinytest - blaze - render - video - muted : !!!!!!!!! FAIL !!!!!!!!!!!
{"groupPath":["tinytest","blaze","render","video"],"test":"muted","events":[{"sequence":10147,"type":"fail","details":{"type":"assert_equal","expected":"false","not":false},"cookie":{"name":"blaze - render - video - muted","offset":0,"groupPath":["tinytest","blaze","render","video"],"shortName":"muted"}}]}
@mitar

This comment has been minimized.

Show comment
Hide comment
@mitar

mitar Feb 25, 2017

Collaborator

Looks good. Thanks.

Collaborator

mitar commented Feb 25, 2017

Looks good. Thanks.

@mitar mitar merged commit 8558cb9 into meteor:master Feb 25, 2017

1 of 2 checks passed

ci/circleci Your tests failed on CircleCI
Details
CLA Author has signed the Meteor CLA.
Details
@abernix

This comment has been minimized.

Show comment
Hide comment
@abernix

abernix Feb 26, 2017

Member

@sethmurphy18 Your tests above seem to have failed spectacularly however the main repo CircleCI seems to be passing with those same commits merged: https://circleci.com/gh/meteor/blaze/tree/master. I don't actually see the specific video test failures you quoted above in the results though.

Anyhow, the Npm.depends should be at the top level of the package.js, not nested within Package.onUse and Package.onTest (though, things will work okay if they are identical in both). See meteor/meteor#8200 (comment) for a more complete explanation.

Things look okay other than that.

Member

abernix commented Feb 26, 2017

@sethmurphy18 Your tests above seem to have failed spectacularly however the main repo CircleCI seems to be passing with those same commits merged: https://circleci.com/gh/meteor/blaze/tree/master. I don't actually see the specific video test failures you quoted above in the results though.

Anyhow, the Npm.depends should be at the top level of the package.js, not nested within Package.onUse and Package.onTest (though, things will work okay if they are identical in both). See meteor/meteor#8200 (comment) for a more complete explanation.

Things look okay other than that.

@mitar

This comment has been minimized.

Show comment
Hide comment
@mitar

mitar Feb 26, 2017

Collaborator

Oh, I missed that. @abernix, thanks for catching that.

@sethmurphy18, could you make another pull request moving Npm.depends to the top?

Collaborator

mitar commented Feb 26, 2017

Oh, I missed that. @abernix, thanks for catching that.

@sethmurphy18, could you make another pull request moving Npm.depends to the top?

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 26, 2017

Contributor

Sure no problem.

Contributor

sethmurphy18 commented Feb 26, 2017

Sure no problem.

@sethmurphy18

This comment has been minimized.

Show comment
Hide comment
@sethmurphy18

sethmurphy18 Feb 26, 2017

Contributor

We are also going to have to update the references to the minifier-js package eventually. It's going to version 2.0.0 which won't be picked up by the html-scanner package since only 1 version of a package can be installed at once and the version constraints won't pick up a new major version.

Contributor

sethmurphy18 commented Feb 26, 2017

We are also going to have to update the references to the minifier-js package eventually. It's going to version 2.0.0 which won't be picked up by the html-scanner package since only 1 version of a package can be installed at once and the version constraints won't pick up a new major version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment