Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reduce code duplication in tests #6409

Open
jeluard opened this issue Feb 8, 2024 · 27 comments
Open

Reduce code duplication in tests #6409

jeluard opened this issue Feb 8, 2024 · 27 comments
Assignees
@FaybianB
Labels
scope-testing Issues for adding test coverage, fixing existing tests or testing strategies.

Comments

@jeluard
Copy link
Member

jeluard commented Feb 8, 2024 

          The amount of copy pasted code is quite high in this file, maybe something that could be improved in this PR (or later one) to make it easier to add new events in the future

Originally posted by @nflaig in #6391 (comment)
@philknows philknows added the scope-testing Issues for adding test coverage, fixing existing tests or testing strategies. label Feb 10, 2024
@FaybianB
Copy link
Contributor

@jeluard @philknows Hey, may I be assigned this issue?

@jeluard
Copy link
Member Author

jeluard commented Feb 21, 2024

Sure!

@FaybianB
Copy link
Contributor

FaybianB commented Feb 26, 2024 •
edited

@jeluard So I'd like to validate that my changes were effective and the tests still pass but I notice that running the e2e tests from unstable (without my changes) fails.

Any advice? Here's the result from yarn test:e2e:

yarn run v1.22.21
$ lerna run test:e2e --concurrency 1
(node:1024) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
lerna notice cli v7.3.0

 >  Lerna (powered by Nx)   Running target test:e2e for 7 projects:

    - @lodestar/beacon-node
    - @chainsafe/lodestar
    - @lodestar/logger
    - @lodestar/params
    - @lodestar/prover
    - @lodestar/spec-test-util
    - @lodestar/validator

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

> @lodestar/params:"test:e2e"

@lodestar/params: $ vitest --run --config vitest.e2e.config.ts --dir test/e2e/
@lodestar/params:  RUN  v1.2.1 /Users/ugongetthisword/Miscellaneous/lodestar/packages/params
@lodestar/params:  ✓ test/e2e/ensure-config-is-synced.test.ts  (2 tests) 611ms
@lodestar/params:  ✓ test/e2e/overridePreset.test.ts  (2 tests) 396ms
@lodestar/params:  ✓ test/e2e/setPreset.test.ts  (2 tests) 368ms
@lodestar/params:  Test Files  3 passed (3)
@lodestar/params:       Tests  6 passed (6)
@lodestar/params:    Start at  21:25:11
@lodestar/params:    Duration  1.60s (transform 56ms, setup 23ms, collect 69ms, tests 1.38s, environment 0ms, prepare 33ms)

> @lodestar/logger:"test:e2e"

@lodestar/logger: $ vitest --run --config vitest.e2e.config.ts --dir test/e2e
@lodestar/logger:  RUN  v1.2.1 /Users/ugongetthisword/Miscellaneous/lodestar/packages/logger
@lodestar/logger: test-log-worker
@lodestar/logger: test-log-worker
@lodestar/logger:  ✓ test/e2e/logger/workerLogs.test.ts  (3 tests) 391ms
@lodestar/logger:  Test Files  1 passed (1)
@lodestar/logger:       Tests  3 passed (3)
@lodestar/logger:    Start at  21:25:14
@lodestar/logger:    Duration  766ms (transform 77ms, setup 17ms, collect 225ms, tests 391ms, environment 0ms, prepare 33ms)

> @lodestar/validator:"test:e2e"

@lodestar/validator: $ vitest --run --config vitest.e2e.config.ts --dir test/e2e
@lodestar/validator:  RUN  v1.2.1 /Users/ugongetthisword/Miscellaneous/lodestar/packages/validator
@lodestar/validator: (node:1088) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
@lodestar/validator: (Use `node --trace-deprecation ...` to show where the warning was created)
@lodestar/validator:  ❯ test/e2e/web3signer.test.ts  (14 tests | 1 skipped) 695ms
@lodestar/validator: ⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯
@lodestar/validator:  FAIL  test/e2e/web3signer.test.ts > web3signer signature test
@lodestar/validator: Error: Could not find a working container runtime strategy
@lodestar/validator:  ❯ getContainerRuntimeClient ../../node_modules/testcontainers/src/container-runtime/clients/client.ts:61:9
@lodestar/validator:  ❯ GenericContainer.start ../../node_modules/testcontainers/src/generic-container/generic-container.ts:81:20
@lodestar/validator:  ❯ Module.startExternalSigner ../test-utils/src/externalSigner.ts:46:28
@lodestar/validator:  ❯ test/e2e/web3signer.test.ts:46:22
@lodestar/validator:      44| 
@lodestar/validator:      45|     const password = "password";
@lodestar/validator:      46|     externalSigner = await startExternalSigner({
@lodestar/validator:        |                      ^
@lodestar/validator:      47|       keystoreStrings: await getKeystoresStr(
@lodestar/validator:      48|         password,
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/2]⎯
@lodestar/validator:  FAIL  test/e2e/web3signer.test.ts > web3signer signature test
@lodestar/validator: TypeError: Cannot read properties of undefined (reading 'container')
@lodestar/validator:  ❯ test/e2e/web3signer.test.ts:61:26
@lodestar/validator:      59| 
@lodestar/validator:      60|   afterAll(async () => {
@lodestar/validator:      61|     await externalSigner.container.stop();
@lodestar/validator:        |                          ^
@lodestar/validator:      62|   });
@lodestar/validator:      63| 
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/2]⎯
@lodestar/validator: ⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯
@lodestar/validator: 
@lodestar/validator: Vitest caught 1 unhandled error during the test run.
@lodestar/validator: This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.
@lodestar/validator: ⎯⎯⎯⎯ Unhandled Rejection ⎯⎯⎯⎯⎯
@lodestar/validator: Error: Cannot find module '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node'
@lodestar/validator: Require stack:
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/bindings.js
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/lib.js
@lodestar/validator:  ❯ Function.Module._resolveFilename node:internal/modules/cjs/loader:1149:15
@lodestar/validator:  ❯ Function.Module._load node:internal/modules/cjs/loader:990:27
@lodestar/validator:  ❯ Module.require node:internal/modules/cjs/loader:1237:19
@lodestar/validator:  ❯ require node:internal/modules/helpers:176:18
@lodestar/validator:  ❯ Object.<anonymous> ../../node_modules/@chainsafe/blst/src/bindings.ts:3:27
@lodestar/validator:  ❯ Module._compile node:internal/modules/cjs/loader:1378:14
@lodestar/validator:  ❯ Object.Module._extensions..js node:internal/modules/cjs/loader:1437:10
@lodestar/validator:  ❯ Module.load node:internal/modules/cjs/loader:1212:32
@lodestar/validator:  ❯ Function.Module._load node:internal/modules/cjs/loader:1028:12
@lodestar/validator:  ❯ Module.require node:internal/modules/cjs/loader:1237:19
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
@lodestar/validator: Serialized Error: { code: 'MODULE_NOT_FOUND', requireStack: [ '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/bindings.js', '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/lib.js' ] }
@lodestar/validator: This error originated in "test/e2e/web3signer.test.ts" test file. It doesn't mean the error was thrown inside the file itself, but while it was running.
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
@lodestar/validator:  Test Files  1 failed (1)
@lodestar/validator:       Tests  1 skipped (14)
@lodestar/validator:      Errors  1 error
@lodestar/validator:    Start at  21:25:15
@lodestar/validator:    Duration  1.69s (transform 448ms, setup 15ms, collect 853ms, tests 695ms, environment 0ms, prepare 32ms)
@lodestar/validator: error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  Lerna (powered by Nx)   Running target test:e2e for 7 projects failed

   Tasks not run because their dependencies failed or --nx-bail=true:
   
   - @lodestar/beacon-node:test:e2e
   - @chainsafe/lodestar:test:e2e
   - @lodestar/prover:test:e2e
   - @lodestar/spec-test-util:test:e2e
   
   Failed tasks:
   
   - @lodestar/validator:test:e2e

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I've tried yarn install to ensure I had all the dependencies but that also fails with a bunch of errors.

@nflaig
Copy link
Member

nflaig commented Feb 26, 2024 

@lodestar/validator: Error: Cannot find module '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node'
@lodestar/validator: Require stack:
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/bindings.js
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/lib.js

Looks like blst is not installed. Did you install all prerequisites noted in MacOS Specifics?

@FaybianB
Copy link
Contributor

Yes, I did all those prerequisites. However, I didn't do these:

Are these necessary to getting the e2e tests running?

./scripts/run_e2e_env.sh start also fails:

Wait for the node to be ready
Waiting for 0.0.0.0:5001.........................................................................................................................
Timeout

@nflaig
Copy link
Member

nflaig commented Feb 26, 2024

Are these necessary to getting the e2e tests running?

No, but first you should make sure the installation (yarn install) doesn't fail and you can build the project (yarn build). From there you can do a sanity check via ./lodestar --help (see Getting Started). If that works you can attempt to run the tests.

./scripts/run_e2e_env.sh start also fails

I remember we recently increased the timeout of this script, and I think the first time it will pull images which might take a bit. Running it a second time should resolve the timeout issue.

I've tried yarn install to ensure I had all the dependencies but that also fails with a bunch of errors.

Can you please share the errors here or on discord (#lodestar-developer channel) which is a better place to figure out what's the issue on your end.

@FaybianB
Copy link
Contributor

@nflaig Thanks, I'd like to ask in the Discord but I haven't found a valid invite (the one you linked is invalid as well).

@nflaig
Copy link
Member

nflaig commented Feb 27, 2024

@nflaig Thanks, I'd like to ask in the Discord but I haven't found a valid invite (the one you linked is invalid as well).

The README invite and this one both work for me. I also newly created the one I linked in the previous comment.

@FaybianB
Copy link
Contributor

@nflaig I'm not sure what the issue is but I get invalid for both.

Screenshot 2024-02-27 at 9 32 45 AM Screenshot 2024-02-27 at 9 33 11 AM

Nonetheless, I'll post the errors here in the meantime:

yarn install v1.22.21
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
[5/5] 🔨  Building fresh packages...
[11/18] ⢀ @chainsafe/blst
[15/18] ⢀ c-kzg
[12/18] ⢀ bcrypto
[14/18] ⢀ cpu-features
error /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst: Command failed.
Exit code: 1
Command: node dist/scripts/install.js
Arguments: 
Directory: /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst
Output:
Retrieving BLST native bindings /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node ...
(node:40510) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Error: Error importing BLST native bindings: 404 Not Found
    at Object.downloadReleaseAsset (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/scripts/downloadReleaseAsset.js:22:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.downloadBindings (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/scripts/downloadBindings.js:9:5)
    at async install (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/scripts/install.js:34:9)
Building BLST native bindings /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node from source...
BLST_WRAP_CPP_PREBUILD /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/blst_wrap.cpp exists, SWIG will be skipped
Launching node-gyp {
  nodeJsExec: '/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node',
  nodeGypExec: '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js',
  cwd: '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js',
  BLST_WRAP_CPP_PREBUILD: '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/blst_wrap.cpp'
}
gyp info it worked if it ends with ok
gyp info using node-gyp@8.4.1
gyp info using node@21.6.1 | darwin | arm64
gyp info find Python using Python version 3.11.7 found at "/opt/homebrew/opt/python@3.11/bin/python3.11"
gyp info spawn /opt/homebrew/opt/python@3.11/bin/python3.11
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  ACTION binding_gyp_blst_target_blst_wrap Release/obj.target/blst/geni/blst_wrap.cpp
/bin/sh: python: command not found
make: *** [Release/obj.target/blst/geni/blst_wrap.cpp] Error 127
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command "/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node" "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js
gyp ERR! node -v v21.6.1
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok 
Error: Error building BLST native bindings: Command failed: /Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js rebuild
gyp info it worked if it ends with ok
gyp info using node-gyp@8.4.1
gyp info using node@21.6.1 | darwin | arm64
gyp info find Python using Python version 3.11.7 found at "/opt/homebrew/opt/python@3.11/bin/python3.11"
gyp info spawn /opt/homebrew/opt/python@3.11/bin/python3.11
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
/bin/sh: python: command not found
make: *** [Release/obj.target/blst/geni/blst_wrap.cpp] Error 127
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command "/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node" "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js
gyp ERR! node -v v21.6.1
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok 

    at genericNodeError (node:internal/errors:984:15)
    at wrappedFn (node:internal/errors:538:14)
    at ChildProcess.exithandler (node:child_process:422:12)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at Socket.<anonymous> (node:internal/child_process:457:11)
    at Socket.emit (node:events:519:28)
    at Pipe.<anonymous> (node:net:337:12) {
  code: 1,
  killed: false,
  signal: null,
  cmd: '/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js rebuild'




warning Error running install script for optional dependency: "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node buildcheck.js > buildcheck.gypi && node-gyp rebuild
Arguments: 
Directory: /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@9.4.1
gyp info using node@21.6.1 | darwin | arm64
gyp info find Python using Python version 3.11.7 found at \"/opt/homebrew/opt/python@3.11/bin/python3.11\"
gyp info spawn /opt/homebrew/opt/python@3.11/bin/python3.11
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_aarch64_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_arm_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_mips_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_ppc_linux.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_freebsd.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_macos.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_windows.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/filesystem.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/stack_line_reader.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/string_view.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/hwcaps.o
  LIBTOOL-STATIC Release/cpu_features.a
  CXX(target) Release/obj.target/cpufeatures/src/binding.o
In file included from ../src/binding.cc:1:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/node.h:73:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8.h:24:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-array-buffer.h:12:
/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-local-handle.h:253:5: error: static_assert failed due to requirement 'std::is_base_of<v8::Value, v8::Data>::value' \"type check\"
    static_assert(std::is_base_of<T, S>::value, \"type check\");
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../nan/nan_callbacks_12_inl.h:175:20: note: in instantiation of function template specialization 'v8::Local<v8::Value>::Local<v8::Data>' requested here
      cbinfo(info, obj->GetInternalField(kDataIndex));
                   ^
1 error generated.
make: *** [Release/obj.target/cpufeatures/src/binding.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/lib/build.js:203:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command \"/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node\" \"/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
gyp ERR! node -v v21.6.1
gyp ERR! node-gyp -v v9.4.1
gyp ERR! not ok"
info This module is OPTIONAL, you can safely ignore this error

@matthewkeil
Copy link
Member

In the error output above one of the native dependencies needs to be built but you are missing python

/bin/sh: python: command not found

Make sure python is on your path and will likely build after that. We are working on getting you into discord now

@matthewkeil
Copy link
Member

What is your discord handle @FaybianB ?

@jeluard
Copy link
Member Author

jeluard commented Feb 27, 2024 •
edited

That's the culprint: /bin/sh: python: command not found.

Before trying to have python detected, could you give this branch jeluard/upgrade-blst a shot?

@nflaig
Copy link
Member

nflaig commented Feb 27, 2024 •
edited

Before trying to have python detected, could you give this branch a shot?

Branch doesn't exist, we merged that change into unstable 3 weeks ago (see #6413)

https://github.com/ChainSafe/lodestar/tree/jeluard/upgrade-blst <-- correct ref to branch

@jeluard
Copy link
Member Author

jeluard commented Feb 27, 2024

My bad, wrong use of markdown :)

@FaybianB
Copy link
Contributor

FaybianB commented Feb 27, 2024 •
edited 

/bin/sh: python: command not found

@matthewkeil So I do have python installed but it's python3. /usr/bin/python does not exist but usr/bin/python3 does. I've tried setting up an alias a few ways:

  • echo "alias python=/usr/bin/python3" >> ~/.zshrc
  • alias python=python3

and although these work in the terminal:

Screenshot 2024-02-27 at 2 53 49 PM

The yarn install fails with the same /bin/sh: python: command not found error. I've spent hours trying to fix this online but I'm stuck.

What is your discord handle @FaybianB ?

ugongetthisword

That's the culprint: /bin/sh: python: command not found.

Before trying to have python detected, could you give this branch jeluard/upgrade-blst a shot?

@jeluard @nflaig Before trying the Python fix above, I checked out this branch, ran yarn install and got the same result.

@jeluard
Copy link
Member Author

jeluard commented Feb 27, 2024

You might want to give a shot at ChainSafe/bls#148 (comment) then.
Or ChainSafe/bls#148 (comment)

@jeluard
Copy link
Member Author

jeluard commented Feb 27, 2024

@FaybianB what is your node version btw?

@jeluard
Copy link
Member Author

jeluard commented Feb 27, 2024

Ok looks like you are using node 21. Can you switch to node 20 and give another shot at branch jeluard/upgrade-blst ?

@FaybianB
Copy link
Contributor

You might want to give a shot at ChainSafe/bls#148 (comment) then. Or ChainSafe/bls#148 (comment)

The first technique yielded some results. yarn install still outputs some errors but it seems to overall have succeeded since subsequent yarn install commands says it's up to date:

ugongetthisword@Workss-MacBook-Pro lodestar % yarn install             
yarn install v1.22.21
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
[5/5] 🔨  Building fresh packages...
[11/18] ⡀ @chainsafe/blst
[-/18] ⡀ waiting...
[12/18] ⡀ bcrypto
[14/18] ⡀ cpu-features
warning Error running install script for optional dependency: "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node buildcheck.js > buildcheck.gypi && node-gyp rebuild
Arguments: 
Directory: /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@9.4.1
gyp info using node@21.6.1 | darwin | arm64
gyp info find Python using Python version 3.12.1 found at \"/opt/homebrew/opt/python@3.12/bin/python3.12\"
gyp info spawn /opt/homebrew/opt/python@3.12/bin/python3.12
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_aarch64_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_arm_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_mips_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_ppc_linux.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_freebsd.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_macos.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_windows.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/filesystem.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/stack_line_reader.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/string_view.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/hwcaps.o
  LIBTOOL-STATIC Release/cpu_features.a
  CXX(target) Release/obj.target/cpufeatures/src/binding.o
In file included from ../src/binding.cc:1:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/node.h:73:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8.h:24:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-array-buffer.h:12:
/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-local-handle.h:253:5: error: static_assert failed due to requirement 'std::is_base_of<v8::Value, v8::Data>::value' \"type check\"
    static_assert(std::is_base_of<T, S>::value, \"type check\");
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../nan/nan_callbacks_12_inl.h:175:20: note: in instantiation of function template specialization 'v8::Local<v8::Value>::Local<v8::Data>' requested here
      cbinfo(info, obj->GetInternalField(kDataIndex));
                   ^
1 error generated.
make: *** [Release/obj.target/cpufeatures/src/binding.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/lib/build.js:203:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command \"/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node\" \"/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
✨  Done in 13.67s.






ugongetthisword@Workss-MacBook-Pro lodestar % yarn install
yarn install v1.22.21
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
success Already up-to-date.
✨  Done in 0.21s.

@FaybianB
Copy link
Contributor

Ok looks like you are using node 21. Can you switch to node 20 and give another shot at branch jeluard/upgrade-blst ?

@jeluard That worked.

Screenshot 2024-02-27 at 3 52 45 PM

@jeluard
Copy link
Member Author

jeluard commented Feb 27, 2024

Thanks for testing!

@FaybianB
Copy link
Contributor

Back on my branch for this ticket:

  • yarn install seems to be passing
  • yarn build succeeds
  • ./lodestar --help successfully displays the help
  • ./scripts/run_e2e_env.sh start continues to timeout. Tried 3 times in a row.
  • yarn test:e2e still fails, assuming because the above fails.

@matthewkeil
Copy link
Member

matthewkeil commented Feb 28, 2024 •
edited 

/bin/sh: python: command not found

@matthewkeil So I do have python installed but it's python3. /usr/bin/python does not exist but usr/bin/python3 does. I've tried setting up an alias a few ways:

  • echo "alias python=/usr/bin/python3" >> ~/.zshrc
  • alias python=python3

and although these work in the terminal:

Screenshot 2024-02-27 at 2 53 49 PM The `yarn install` fails with the same `/bin/sh: python: command not found` error. I've spent hours trying to fix this online but I'm stuck.

Unfortunately one of the dependencies of one of our dependencies has a hard coded (in an internal script) call to python and that gets run in a subshell that does not load dotfiles which is why you are seeing the alias work on the command line but not during installation. The only way around this is to use pyenv as it puts itself natively on PATH and then proxies the call to the appropriate python version (replacing python for python3 if necessary).

@jeluard
Copy link
Member Author

jeluard commented Feb 28, 2024

@matthewkeil That part is fixed now
@FaybianB You need to have docker running for ./scripts/run_e2e_env.sh start to work

@matthewkeil
Copy link
Member

matthewkeil commented Feb 28, 2024 •
edited

@FaybianB I sent a friend request to you on Discord. I will be able to manually add you to the server from there. Please feel free to DM me when you see it and we will get you settled in. Welcome!

@jeluard jeluard mentioned this issue Feb 28, 2024
Merged
@FaybianB
Copy link
Contributor

@matthewkeil That part is fixed now @FaybianB You need to have docker running for ./scripts/run_e2e_env.sh start to work

So do I need to do that in order to run the end-to-end tests?

@jeluard
Copy link
Member Author

jeluard commented Feb 28, 2024

You need to have docker running, yes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@FaybianB FaybianB

Labels
scope-testing Issues for adding test coverage, fixing existing tests or testing strategies.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@jeluard @matthewkeil @nflaig @FaybianB @philknows