Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: cache native npm modules during docker build #20

Merged
merged 8 commits into from
Nov 9, 2021

Conversation

shumkov
Copy link
Member

@shumkov shumkov commented Nov 8, 2021

Issue being fixed or feature implemented

There are packages that uses node gyp to build native modules. It takes decent amount of time. Previously we had a solution to cache results: https://github.com/frangio/node-gyp-cache

After migration to monorepo, Node 16 and NPM 8, this solution doesn't work anymore.

What was done?

  • Removed node-gyp-cache
  • Built slow modules in additional stage and copy the results to eventual node modules dir
  • Removed dashd-zmq-client from Drive

How Has This Been Tested?

Running manually

Breaking Changes

None

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

@shumkov shumkov added this to the v0.21.x milestone Nov 8, 2021
@shumkov shumkov changed the title build: enable npn build: enable npm cache for native modules Nov 8, 2021
@shumkov shumkov changed the title build: enable npm cache for native modules build: cache native npm modules during docker build Nov 8, 2021
@shumkov shumkov requested a review from strophy November 8, 2021 10:16
@strophy
Copy link
Collaborator

strophy commented Nov 8, 2021

Can we run this a few times to test setup times in CI? I think we will still need to remove npm run clean:data from the setup script or add back layer caching instead of (or as well as) cache mounts.

antouhou
antouhou previously approved these changes Nov 8, 2021
@shumkov
Copy link
Member Author

shumkov commented Nov 8, 2021

@strophy yeah, I ran it many times and if everything is cached it takes seconds to build. I've updated clean:data and going to push it as a separate PR.

strophy
strophy previously approved these changes Nov 8, 2021
Copy link
Collaborator

@strophy strophy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

antouhou
antouhou previously approved these changes Nov 8, 2021
Copy link
Collaborator

@strophy strophy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Awesome fix! 🥇

@shumkov shumkov merged commit a9a5a1b into master Nov 9, 2021
@shumkov shumkov deleted the docker-npm-cache branch November 9, 2021 10:29
shumkov pushed a commit that referenced this pull request Nov 23, 2022
…lidation

feat: documents batch transition - stateless validation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants