New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

METRON-1480 Add yarn as default build tool for the frontend #952

Closed
wants to merge 27 commits into
base: master
from

Conversation

Projects
None yet
7 participants
@xyztdanid4
Contributor

xyztdanid4 commented Mar 7, 2018

Updated the build tool to yarn for the frontend project to avoid npm instability during dependency adding.

For all changes:

  • Is there a JIRA ticket associated with this PR? If not one needs to be created at https://issues.apache.org/jira/browse/METRON-1480.
  • Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
  • Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

  • Have you included steps to reproduce the behavior or problem that is being changed or addressed?

  • Have you included steps or a guide to how the change may be verified and tested manually?

  • Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:

    mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh 
    
  • Have you written or updated unit tests and or integration tests to verify your changes?

  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

  • Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

  • Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:

    cd site-book
    mvn site
    

Note:

Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

@simonellistonball

This comment has been minimized.

Show comment
Hide comment
@simonellistonball

simonellistonball Mar 7, 2018

Contributor

Have the dependencies, License and Notices files been updated?

Contributor

simonellistonball commented Mar 7, 2018

Have the dependencies, License and Notices files been updated?

@xyztdanid4

This comment has been minimized.

Show comment
Hide comment
@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

@simonellistonball I am not sure. Probably no.

Contributor

xyztdanid4 commented Mar 7, 2018

@simonellistonball I am not sure. Probably no.

@@ -1,6427 +0,0 @@
{

This comment has been minimized.

@cestella

cestella Mar 7, 2018

Member

I suspect the build failure is due to a lack of a license on this file. Since JSON cannot take comments, you'll need to adjust the top level pom to add an exclusion for **/package-lock.json here Just make a quick comment above it and note that this is yarn collateral.

@cestella

cestella Mar 7, 2018

Member

I suspect the build failure is due to a lack of a license on this file. Since JSON cannot take comments, you'll need to adjust the top level pom to add an exclusion for **/package-lock.json here Just make a quick comment above it and note that this is yarn collateral.

This comment has been minimized.

@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

I think that was the problem!
Thanks. @cestella

@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

I think that was the problem!
Thanks. @cestella

@xyztdanid4 xyztdanid4 changed the title from Metron-1480 Add yarn as default build tool for the frontend to METRON-1480 Add yarn as default build tool for the frontend Mar 7, 2018

@@ -33,6 +28,11 @@
"@angular/platform-browser": "2.0.0",
"@angular/platform-browser-dynamic": "2.0.0",
"@angular/router": "3.0.0",
"@types/ace": "0.0.32",

This comment has been minimized.

@merrimanr

merrimanr Mar 7, 2018

Contributor

Were there any real changes made to this file? It looks like some dependencies were simply rearranged. I do see the addition of "main": "index.js". What is the purpose of that?

@merrimanr

merrimanr Mar 7, 2018

Contributor

Were there any real changes made to this file? It looks like some dependencies were simply rearranged. I do see the addition of "main": "index.js". What is the purpose of that?

This comment has been minimized.

@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

No there were no changes other than rearrange of the dependencies.
About the main attribute, in our case it will do nothing. It defines the entry point of the application if we would SystemJS or requireJS. It is the default setting for yarn.

@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

No there were no changes other than rearrange of the dependencies.
About the main attribute, in our case it will do nothing. It defines the entry point of the application if we would SystemJS or requireJS. It is the default setting for yarn.

This comment has been minimized.

@merrimanr

merrimanr Mar 7, 2018

Contributor

What was the reason for rearranging the dependencies? It's not a big deal but it's distracting and makes reviewing harder.

@merrimanr

merrimanr Mar 7, 2018

Contributor

What was the reason for rearranging the dependencies? It's not a big deal but it's distracting and makes reviewing harder.

This comment has been minimized.

@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

Yes. I was not enough carefully. Yarn did the rearrangement, but I can move the dependencies back to their original place if you would like to.

@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

Yes. I was not enough carefully. Yarn did the rearrangement, but I can move the dependencies back to their original place if you would like to.

This comment has been minimized.

@merrimanr

merrimanr Mar 7, 2018

Contributor

No I was just curious. If Yarn did this automatically we might as well leave it that way. No need to revert.

@merrimanr

merrimanr Mar 7, 2018

Contributor

No I was just curious. If Yarn did this automatically we might as well leave it that way. No need to revert.

"ng2-dragula": "^1.5.0",
"pikaday-time": "^1.6.1",

This comment has been minimized.

@merrimanr

merrimanr Mar 7, 2018

Contributor

See comment for metron-config/package.json.

@merrimanr

merrimanr Mar 7, 2018

Contributor

See comment for metron-config/package.json.

@merrimanr

This comment has been minimized.

Show comment
Hide comment
@merrimanr

merrimanr Mar 7, 2018

Contributor

I'm having trouble understanding what this PR actually does. I see that package-lock.json was removed and yarn.lock was added. Is that all that's needed to switch to yarn? Do we need to also update our build commands to match those here?

What considerations are there for development? Are we supposed to be using yarn commands now instead of npm commands? Do we need to install yarn locally? We probably need to review our documentation and rpm build processes too. We use npm all over the place.

Contributor

merrimanr commented Mar 7, 2018

I'm having trouble understanding what this PR actually does. I see that package-lock.json was removed and yarn.lock was added. Is that all that's needed to switch to yarn? Do we need to also update our build commands to match those here?

What considerations are there for development? Are we supposed to be using yarn commands now instead of npm commands? Do we need to install yarn locally? We probably need to review our documentation and rpm build processes too. We use npm all over the place.

@xyztdanid4

This comment has been minimized.

Show comment
Hide comment
@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

@merrimanr
Yes this is the right way to switch to yarn.
Yes we have install yarn locally.
I did not know that we have so much dependency on npm. But for me it was hard to add new dependencies using npm, cause it was constantly failing. And yes I would recommend to use yarn, at least for the front end builds. What yarn does is that reduces the possibility of version mismatch of the dependencies by importing the dependencies subdependency in the right version. So yes we have to use yarn commands if we merge this PR.

Contributor

xyztdanid4 commented Mar 7, 2018

@merrimanr
Yes this is the right way to switch to yarn.
Yes we have install yarn locally.
I did not know that we have so much dependency on npm. But for me it was hard to add new dependencies using npm, cause it was constantly failing. And yes I would recommend to use yarn, at least for the front end builds. What yarn does is that reduces the possibility of version mismatch of the dependencies by importing the dependencies subdependency in the right version. So yes we have to use yarn commands if we merge this PR.

@xyztdanid4

This comment has been minimized.

Show comment
Hide comment
@xyztdanid4

xyztdanid4 Mar 7, 2018

Contributor

@merrimanr
although it should work with npm as well, cause then it will use the package.json to install the dependencies.

Contributor

xyztdanid4 commented Mar 7, 2018

@merrimanr
although it should work with npm as well, cause then it will use the package.json to install the dependencies.

@merrimanr

This comment has been minimized.

Show comment
Hide comment
@merrimanr

merrimanr Mar 7, 2018

Contributor

I think we are on board with switching to yarn so no concerns there. There was a discussion on it but if anyone now feels differently, speak up.

Sounds like we still have a lot of work to do. I would suggest reviewing the metron-config and metron-alerts READMEs and update them appropriately. In fact, I would just search the whole project for npm commands and switch to yarn as needed. You will definitely need to run this up in full dev and verify everything works.

Our maven front end plugin will also need to be reviewed and updated: https://github.com/eirslett/frontend-maven-plugin. We use that in metron-config and metron-alerts.

Contributor

merrimanr commented Mar 7, 2018

I think we are on board with switching to yarn so no concerns there. There was a discussion on it but if anyone now feels differently, speak up.

Sounds like we still have a lot of work to do. I would suggest reviewing the metron-config and metron-alerts READMEs and update them appropriately. In fact, I would just search the whole project for npm commands and switch to yarn as needed. You will definitely need to run this up in full dev and verify everything works.

Our maven front end plugin will also need to be reviewed and updated: https://github.com/eirslett/frontend-maven-plugin. We use that in metron-config and metron-alerts.

@xyztdanid4

This comment has been minimized.

Show comment
Hide comment
@xyztdanid4

xyztdanid4 Mar 9, 2018

Contributor

@merrimanr
As we agreed yesterday I did the required changes in the pom xml-s.
I had to update the node dependency of the projects, otherwise the build was failed, cause one of the dependencies was dependent on node8.
There is no equivalent command to npm --prefix in yarn, so I updated the readme to just cd into that directory where the install step is required.
When we build the metron-alert project (angular4), there was an error, mainly because angular4 uses AOT as default builder strategy. I fixed this by addig env=prod to the package json.

Contributor

xyztdanid4 commented Mar 9, 2018

@merrimanr
As we agreed yesterday I did the required changes in the pom xml-s.
I had to update the node dependency of the projects, otherwise the build was failed, cause one of the dependencies was dependent on node8.
There is no equivalent command to npm --prefix in yarn, so I updated the readme to just cd into that directory where the install step is required.
When we build the metron-alert project (angular4), there was an error, mainly because angular4 uses AOT as default builder strategy. I fixed this by addig env=prod to the package json.

</goals>
<configuration>
<arguments>install</arguments>
<arguments>install --mutex network --ignore-engines --pure-lockfile</arguments>

This comment has been minimized.

@xyztdanid4

xyztdanid4 Mar 12, 2018

Contributor

@merrimanr I would like to thank you for your help with pointing out the problem is with the CI. I tried to set the cache folder differently for each angular project, but somehow I got some permission errors. The solution is that I introduced the 'mutex network' flag on the yarn install, so it will prevent multiple installation on the same time by yarn.

@xyztdanid4

xyztdanid4 Mar 12, 2018

Contributor

@merrimanr I would like to thank you for your help with pointing out the problem is with the CI. I tried to set the cache folder differently for each angular project, but somehow I got some permission errors. The solution is that I introduced the 'mutex network' flag on the yarn install, so it will prevent multiple installation on the same time by yarn.

@sardell

This comment has been minimized.

Show comment
Hide comment
@sardell

sardell May 15, 2018

Contributor

What yarn does is that reduces the possibility of version mismatch of the dependencies by importing the dependencies subdependency in the right version

I believe npm also does this with the addition of package-lock.json.

Contributor

sardell commented May 15, 2018

What yarn does is that reduces the possibility of version mismatch of the dependencies by importing the dependencies subdependency in the right version

I believe npm also does this with the addition of package-lock.json.

@nickwallen

This comment has been minimized.

Show comment
Hide comment
@nickwallen

nickwallen May 15, 2018

Contributor

@sardell Thanks.

@xyztdanid4 Please let us know if you feel this PR is still relevant. If not, please close the PR. Otherwise, the PR may be closed as inactive according to the Metron Development Guidelines.

Contributor

nickwallen commented May 15, 2018

@sardell Thanks.

@xyztdanid4 Please let us know if you feel this PR is still relevant. If not, please close the PR. Otherwise, the PR may be closed as inactive according to the Metron Development Guidelines.

@nickwallen

This comment has been minimized.

Show comment
Hide comment
@nickwallen

nickwallen May 29, 2018

Contributor

-1 as this has become an abandoned PR. I will open a request to close this.

Contributor

nickwallen commented May 29, 2018

-1 as this has become an abandoned PR. I will open a request to close this.

@pono pono closed this May 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment