-
-
Notifications
You must be signed in to change notification settings - Fork 290
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: add distutils to build phase of Dockerfile #6845
Conversation
@@ -23,7 +23,7 @@ RUN cd packages/cli && GIT_COMMIT=${COMMIT} yarn write-git-data | |||
# Note: This step is redundant for the host arch | |||
FROM node:22-alpine as build_deps | |||
WORKDIR /usr/app | |||
RUN apk update && apk add --no-cache g++ make python3 && rm -rf /var/cache/apk/* | |||
RUN apk update && apk add --no-cache g++ make python3 py3-setuptools && rm -rf /var/cache/apk/* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
but is also necessary for build phase
docker build works fine on unstable, what are the requirements for adding this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was throwing an error for missing distutils when I built locally for debugging the performance regression. Not sure why it didnt work but it didnt...
70.95 (node:97) [DEP0060] DeprecationWarning: The `util._extend` API is deprecated. Please use Object.assign() instead.
70.95 (Use `node --trace-deprecation ...` to show where the warning was created)
70.95 gyp info find Python using Python version 3.12.3 found at "/usr/bin/python3"
70.95 gyp http GET https://nodejs.org/download/release/v22.2.0/node-v22.2.0-headers.tar.gz
70.95 gyp http 200 https://nodejs.org/download/release/v22.2.0/node-v22.2.0-headers.tar.gz
70.95 gyp http GET https://nodejs.org/download/release/v22.2.0/SHASUMS256.txt
70.95 gyp http 200 https://nodejs.org/download/release/v22.2.0/SHASUMS256.txt
70.95 gyp info spawn /usr/bin/python3
70.95 gyp info spawn args [
70.95 gyp info spawn args '/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py',
70.95 gyp info spawn args 'binding.gyp',
70.95 gyp info spawn args '-f',
70.95 gyp info spawn args 'make',
70.95 gyp info spawn args '-I',
70.95 gyp info spawn args '/usr/app/node_modules/bcrypto/build/config.gypi',
70.95 gyp info spawn args '-I',
70.95 gyp info spawn args '/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/addon.gypi',
70.95 gyp info spawn args '-I',
70.95 gyp info spawn args '/root/.cache/node-gyp/22.2.0/include/node/common.gypi',
70.95 gyp info spawn args '-Dlibrary=shared_library',
70.95 gyp info spawn args '-Dvisibility=default',
70.95 gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/22.2.0',
70.95 gyp info spawn args '-Dnode_gyp_dir=/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp',
70.95 gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/22.2.0/<(target_arch)/node.lib',
70.95 gyp info spawn args '-Dmodule_root_dir=/usr/app/node_modules/bcrypto',
70.95 gyp info spawn args '-Dnode_engine=v8',
70.95 gyp info spawn args '--depth=.',
70.95 gyp info spawn args '--no-parallel',
70.95 gyp info spawn args '--generator-output',
70.95 gyp info spawn args 'build',
70.95 gyp info spawn args '-Goutput_dir=.'
70.95 gyp info spawn args ]
70.95 Traceback (most recent call last):
70.95 File "/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py", line 42, in <module>
70.95 import gyp # noqa: E402
70.95 ^^^^^^^^^^
70.95 File "/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 9, in <module>
70.95 import gyp.input
70.95 File "/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/pylib/gyp/input.py", line 19, in <module>
70.95 from distutils.version import StrictVersion
70.95 ModuleNotFoundError: No module named 'distutils'
70.95 gyp ERR! configure error
70.95 gyp ERR! stack Error: `gyp` failed with exit code: 1
70.95 gyp ERR! stack at ChildProcess.onCpExit (/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/lib/configure.js:259:16)
70.95 gyp ERR! stack at ChildProcess.emit (node:events:520:28)
70.95 gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
70.95 gyp ERR! System Linux 6.3.13-linuxkit
70.95 gyp ERR! command "/usr/local/bin/node" "/usr/app/node_modules/.bin/node-gyp" "rebuild"
70.95 gyp ERR! cwd /usr/app/node_modules/bcrypto
70.95 gyp ERR! node -v v22.2.0
70.95 gyp ERR! node-gyp -v v8.4.1
70.95 gyp ERR! not ok
70.95 info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
71.21 info This module is OPTIONAL, you can safely ignore this error
71.21 warning Error running install script for optional dependency: "/usr/app/node_modules/cpu-features: Command failed.
71.21 Exit code: 1
71.21 Command: node buildcheck.js > buildcheck.gypi && node-gyp rebuild
71.21 Arguments:
71.21 Directory: /usr/app/node_modules/cpu-features
71.21 Output:
71.21 gyp info it worked if it ends with ok
71.21 gyp info using node-gyp@8.4.1
71.21 gyp info using node@22.2.0 | linux | arm64
71.21 (node:126) [DEP0060] DeprecationWarning: The `util._extend` API is deprecated. Please use Object.assign() instead.
71.21 (Use `node --trace-deprecation ...` to show where the warning was created)
71.21 gyp info find Python using Python version 3.12.3 found at \"/usr/bin/python3\"
71.21 gyp http GET https://nodejs.org/download/release/v22.2.0/node-v22.2.0-headers.tar.gz
71.21 gyp http 200 https://nodejs.org/download/release/v22.2.0/node-v22.2.0-headers.tar.gz
71.21 gyp http GET https://nodejs.org/download/release/v22.2.0/SHASUMS256.txt
71.21 gyp http 200 https://nodejs.org/download/release/v22.2.0/SHASUMS256.txt
71.21 gyp info spawn /usr/bin/python3
71.21 gyp info spawn args [
71.21 gyp info spawn args '/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py',
71.21 gyp info spawn args 'binding.gyp',
71.21 gyp info spawn args '-f',
71.21 gyp info spawn args 'make',
71.21 gyp info spawn args '-I',
71.21 gyp info spawn args '/usr/app/node_modules/cpu-features/build/config.gypi',
71.21 gyp info spawn args '-I',
71.21 gyp info spawn args '/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/addon.gypi',
71.21 gyp info spawn args '-I',
71.21 gyp info spawn args '/root/.cache/node-gyp/22.2.0/include/node/common.gypi',
71.21 gyp info spawn args '-Dlibrary=shared_library',
71.21 gyp info spawn args '-Dvisibility=default',
71.21 gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/22.2.0',
71.21 gyp info spawn args '-Dnode_gyp_dir=/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp',
71.21 gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/22.2.0/<(target_arch)/node.lib',
71.21 gyp info spawn args '-Dmodule_root_dir=/usr/app/node_modules/cpu-features',
71.21 gyp info spawn args '-Dnode_engine=v8',
71.21 gyp info spawn args '--depth=.',
71.21 gyp info spawn args '--no-parallel',
71.21 gyp info spawn args '--generator-output',
71.21 gyp info spawn args 'build',
71.21 gyp info spawn args '-Goutput_dir=.'
71.21 gyp info spawn args ]
71.21 Traceback (most recent call last):
71.21 File \"/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py\", line 42, in <module>
71.21 import gyp # noqa: E402
71.21 ^^^^^^^^^^
71.21 File \"/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/pylib/gyp/__init__.py\", line 9, in <module>
71.21 import gyp.input
71.21 File \"/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/pylib/gyp/input.py\", line 19, in <module>
71.21 from distutils.version import StrictVersion
71.21 ModuleNotFoundError: No module named 'distutils'
71.21 gyp ERR! configure error
71.21 gyp ERR! stack Error: `gyp` failed with exit code: 1
71.21 gyp ERR! stack at ChildProcess.onCpExit (/usr/app/node_modules/@chainsafe/blst/node_modules/node-gyp/lib/configure.js:259:16)
71.21 gyp ERR! stack at ChildProcess.emit (node:events:520:28)
71.21 gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
71.21 gyp ERR! System Linux 6.3.13-linuxkit
71.21 gyp ERR! command \"/usr/local/bin/node\" \"/usr/app/node_modules/.bin/node-gyp\" \"rebuild\"
71.21 gyp ERR! cwd /usr/app/node_modules/cpu-features
71.21 gyp ERR! node -v v22.2.0
71.21 gyp ERR! node-gyp -v v8.4.1
71.21 gyp ERR! not ok"
------
Dockerfile:32
--------------------
30 | # Do yarn --force to trigger a rebuild of the native packages
31 | # Emmulates `yarn rebuild` which is not available in v1 https://yarnpkg.com/cli/rebuild
32 | >>> RUN yarn install --non-interactive --frozen-lockfile --production --force
33 | # Rebuild leveldb bindings (required for arm64 build)
34 | RUN cd node_modules/classic-level && yarn rebuild
--------------------
ERROR: failed to solve: process "/bin/sh -c yarn install --non-interactive --frozen-lockfile --production --force" did not complete successfully: exit code: 1
Performance Report✔️ no performance regression detected Full benchmark results
|
* chore: n-historical state - track state persist count per epoch (#6699) * feat: add lodestar binaries to release (#6666) * feat: added binaries creation support * feat: added sanity checks * chore: define reusable workflow * chore: fix sanity check,; disable arm64 * chore: added workflow_dispatch * fix: needs cleanup * chore: update old space size Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: address comments * chore: address comments * chore: fixed typo * chore: typo Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: address comments * chore: wording Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: use frozen lock file Co-authored-by: Nico Flaig <nflaig@protonmail.com> --------- Co-authored-by: Nico Flaig <nflaig@protonmail.com> * feat: switch bls to napi rebuild (#6616) * chore: update package.json deps and run yarn * feat: remove unused blst config from vitest.browser.*.config.ts * test: move G2@INF test to base blst repo. No longer exported. * fix: add Uint8Array.from for Buffer return value in unit test so type matches for toEqual * refactor: update imports of CoordType to use bls instead of blst * feat: add blsAddVerificationRandomness cli option * feat: add randomBytesNonZero to utils * feat: implement verification randomness for same message * feat: implement napi blst and switch to libuv worker pool * chore: lint * fix: update comments for availableParallelism * feat: set UV_THREADPOOL_SIZE for perf tests * feat: set UV_THREADPOOL_SIZE=$(nproc) in lodestar entrance script * feat: remove case for serialized PublicKey in deserializeSet * feat: add back workBusy count logic to canAcceptWork * feat: automatically set threadpool size in beaconHandler * refactor: rename flag to disableSameMessageVerificationRandomness * fix: remove UV_THREADPOOL_SIZE from ./lodestar * docs: add TODO about updating metrics * fix: make benchmark multi threading cross-compatible * chore: add bls@8.0.0 * feat: default to added randomness * feat: add warning for no same message randomness * fix: passing to updated bls functions * feat: add blst peerDep * fix: default to same message randomness * chore: update yarn.lock * chore: remove unused function to clear lint * feat: break out setThreadPoolSize * docs: update packages/cli/src/cmds/beacon/setThreadPoolSize.ts Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: update location for blst dep * chore: run lint again * fix: remove peerDep from light-client * test: fix browser based tests * fix: bad relative imports * docs: add note to readme about switchable bls * docs: typo * chore: lint readme * fix: light-client unit tests * fix: light-client e2e test * feat: remove disableSameMessageVerificationRandomness * refactor: remove unused type * docs: update wordlist * fix: e2e tests * Lint package.json * Revert "fix: e2e tests" This reverts commit f71acd7. * Revert "fix: light-client e2e test" This reverts commit 23c11f0. * Revert "fix: light-client unit tests" This reverts commit fa7893e. * Revert "test: fix browser based tests" This reverts commit 98d04ab. * docs: update readme to remove switchable bls argument * fix: replace optimizeDeps in vitest.browser.config * chore: updated to bls 8.1.0 * feat: move setting threadpool size and warnings to applyPreset.ts * fix: remove setThreadpoolSize from beacon handler * feat: move libuv size logging into handler to use logger * refactor: rename applyPreset to preInitialization * refactor: logging about pool size * Update packages/cli/src/options/globalOptions.ts Co-authored-by: Nico Flaig <nflaig@protonmail.com> * refactor: rename prover applyPreset * docs: change comment in cli/index * fix: update logging for * fix: throw error for invalid uvThreadpoolSize * Update packages/cli/src/preInitialization.ts Co-authored-by: Nico Flaig <nflaig@protonmail.com> * Fix lint issues * Remove unused import * refactor: move bls pool logging back to multithread.ts * fix: log of threadcount --------- Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: update references to consensus spec to v1.4.0 (#6704) * chore: update references to consensus spec to v1.4.0 * Remove unintended formatting changes * chore: update consensus spec version in README (#6705) * docs: correct plausible url (#6709) fix: correct plausible url * docs: incorrect scripts location (#6710) fix: incorrect scripts location * docs: no more broken images (#6708) * fix: no more broken images * fix: incorrect paths * feat: check aggregationBits length of `SignedAggregateAndProof` in gossip validation (#6700) * Add unit test * Update packages/beacon-node/src/chain/validation/aggregateAndProof.ts Co-authored-by: Nico Flaig <nflaig@protonmail.com> --------- Co-authored-by: Nico Flaig <nflaig@protonmail.com> * feat: add and track blob source for metrics (#6628) * feat: add and track blob source for metrics * adding blobsSource for blobsPromise * address feedback * apply feedback * updates * only allow non blobspromise blocks to be imported * feat: add support for browser bundle for lightclient (#6673) * Reorganize the code so it is accessible from one package * Add support for browser build for lightclient * Update the build config * Improve the bls vite plugin * Restructure the vite and vitest scripts * Simplify vite config * Remove unused polyfill * Fix the doc lint error * Add support for bundle test * Update the package json files * Add dist build to default build task * Fix spelling in the docs * Fix the lint error * Fix type error * Disable eslint errors * Increase the timeout for bundle test * Fix eslint bundle * Fix lint warning * Remove the unused config * Add the default export to bundle * Enable compression on th build * Update packages/light-client/README.md Co-authored-by: Nico Flaig <nflaig@protonmail.com> * Increase timeout for one test * Optimize package build task * Update the readme --------- Co-authored-by: Nico Flaig <nflaig@protonmail.com> * docs: fix plausible domain (#6712) * ci: add paths-ignore to workflows (#6713) * chore: added paths-ignore * chore: address comments * feat: add arm64 binary support (#6707) * feat: add arm64 binary support * chore: workaround yarn * chore: add comment * chore: workaround runner limitations * ci: publish binaries for release candidates (#6714) * feat: add types support to esm bundle for lightclient (#6685) * Add vite dts plugin * Restructure types export * Update vite build config * Remove unused file * Update the build process * Fix doc formatting * Build the dist for type tests * chore: update vitest (#6718) Update vitest * chore: do not build all binaries atomically (#6719) * feat: update libp2p deps (#6711) * feat: update libp2p deps * chore: fix linter/type errors * chore(deps): bump ejs from 3.1.9 to 3.1.10 (#6721) Bumps [ejs](https://github.com/mde/ejs) from 3.1.9 to 3.1.10. - [Release notes](https://github.com/mde/ejs/releases) - [Commits](mde/ejs@v3.1.9...v3.1.10) --- updated-dependencies: - dependency-name: ejs dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: revert "ci: add paths-ignore to workflows (#6713)" (#6730) Revert "ci: add paths-ignore to workflows (#6713)" This reverts commit e460bb1. * docs: restore docs pages lost during docusaurus migration (#6725) * ci: fix incorrect build caching (#6731) fix: incorrect build caching * fix: build error log during esm bundle build (#6737) Fix build error * chore: update fastify to v4.27.0 (#6740) * deps: upgrade vitest minor version (#6745) Upgrade vitest * chore: warn when heap limit is too low (#6722) * chore: warn when heap limit is too low * chore: units Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: address comments * chore: added faq element * chore: address comments Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: address comments Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: address comments --------- Co-authored-by: Nico Flaig <nflaig@protonmail.com> * feat: disable flood publish (#6755) * feat: disable flood publish * feat: add network.disableFloodPublish cli flag * chore: address PR comments * test: allow compact diff for test failures (#6783) * Allow the compact diff for tests * Update the doc * Update docs/pages/contribution/testing/index.md Co-authored-by: Julien <jeluard@users.noreply.github.com> * Fix linting on docs * Fix linting --------- Co-authored-by: Julien <jeluard@users.noreply.github.com> * chore: upgrade classic-level (#6793) * chore: upgrade classic-level * chore: address comments * feat(prover): support non-mutated verification provider in prover (#6727) * Restructure the assertions * Add an inspector to run the logic to detect providers * Update web3 provdier logic to use inspetor * Fix the types for proxy * Make the default type for mutation * Rename elrpc to elrpcprovider * Apply suggestions from code review Co-authored-by: Julien <jeluard@users.noreply.github.com> * Fix build error * Update the readme doc * Apply suggestions from code review Co-authored-by: Julien <jeluard@users.noreply.github.com> * Fix the docs linting * Add missing words --------- Co-authored-by: Julien <jeluard@users.noreply.github.com> * docs: added debugging section (#6743) * docs: added debugging section * chore: spell checks * chore: added extra docs * chore: address comments * chore: added extra configuration * chore: updated docs * chore: added extra configuration * chore: fix lint * chore: fix typos * chore: .gitignore * chore: address comments * chore: address comments * chore: review debugging section PR (#6807) * chore: update the naming conventions used in sim tests (#6805) * Rename simulation test to crucible * Rename SimulationEnvironment to Simulation * Use consistent function names * Update readme * Rename interfaces for consistent pattern * Fix lint error * feat: recognize grandine from agentVersion (#6808) * Add grandine * add grandine to wordlist * chore: refactor event loop interactions (#6806) * chore: refactor sleep(0) usage * chore: refactor setTimeout * chore: address comments * chore: cleanup * Apply suggestions from code review --------- Co-authored-by: Cayman <caymannava@gmail.com> * chore: fix broken docker (#6813) * chore: log newPayload engine api (#6810) * chore: add `get_proposer_head` check in fork choice spec test (#6814) * Check get_proposer_head * Fix enum typo * lint * chore: consolidate classic-level usage (#6795) * chore: do not rely on leveldown * chore: replace level with classic-level * chore: added docker support for osx (#6696) * chore: added docker support for osx * chore: address comments * chore: address comments * Update docker-compose.yml Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: address comments --------- Co-authored-by: Nico Flaig <nflaig@protonmail.com> * chore: fix vitest performance with spec tests artifacts in place (#6817) Fix vitest performance with spec tests in place * chore: clean up yarn lockfile (#6816) * feat: disable flood publish by default (#6815) * feat: use @chainsafe/blst directly (#6706) * feat: use @chainsafe/blst directly * chore: update to blst@1.0.1 * refactor: remove randomBytesNonZero and user blst exported version * chore: update blst references * test: catch invalid deserialization in spec tests and return false * feat: create signatureFromBytes and signatureFromBytesNoCheck in utils package * feat: implement signatureFromBytes from utils package * feat: implement signatureFromBytes everywhere * fix: light-client empty module for blst --------- Co-authored-by: matthewkeil <me@matthewkeil.com> * fix: avoid Buffer.from copies (#6723) * fix: avoid Buffer.from copies * chore: simplify shuffling * fix: use subarray instead of slice in shuffling * chore: remove unnecessary devDependencies * chore: rely on fastify 4.x behavior * chore: avoid copy in verifyMerkleBranch * use toBase64 * relax assertions in shuffle function * Update packages/state-transition/src/util/shuffle.ts Co-authored-by: twoeths <tuyen@chainsafe.io> --------- Co-authored-by: twoeths <tuyen@chainsafe.io> * chore: update nodejs to latest 22 (#6729) * Upgrade node js version to 22 * Update node types * Revert action config * Add package from git hash * Fix the build error * Update the docs for node-22 * Update docker version for Nodejs to 22 * Update package.json * Update package.json Co-authored-by: Nico Flaig <nflaig@protonmail.com> * Update readme docs * Add word in dictionary * Add word in dictionary --------- Co-authored-by: Cayman <caymannava@gmail.com> Co-authored-by: Nico Flaig <nflaig@protonmail.com> * deps: update systeminformation to 5.22.9 (#6823) * chore: add distutils to build phase of Dockerfile (#6845) * test: increase timeout of keystore tests (#6846) * test: increase timeout of keystore cache tests * Increase hook timeout * Consistent number formatting * chore: only log warning if media type is not supported (415) (#6847) * fix: revert napi bls (#6853) * Revert "feat: use @chainsafe/blst directly (#6706)" This reverts commit 66fe753. * Revert "feat: switch bls to napi rebuild (#6616)" This reverts commit 97d9aa8. * chore: bump package versions to 1.19.0 * chore: bump @chainsafe/blst to v0.2.11 (#6856) * fix: arm64 binaries creation issue (#6858) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: tuyennhv <tuyen@chainsafe.io> Co-authored-by: Julien <jeluard@users.noreply.github.com> Co-authored-by: Nico Flaig <nflaig@protonmail.com> Co-authored-by: Matthew Keil <me@matthewkeil.com> Co-authored-by: NC <adrninistrator1@protonmail.com> Co-authored-by: g11tech <develop@g11tech.io> Co-authored-by: Nazar Hussain <nazarhussain@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
🎉 This PR is included in v1.19.0 🎉 |
Motivation
Python 3.12 does not come with
distutils
by default any more and its required bynode-gyp
. Was added to install phase but is also necessary for build phase