Apple Silicon M1 support?

[Sajjon]

I would like to know if Lodestar supports Apple Silicon M1 chip?

[dapplion]

The limiting factor may be our BLST crypto lib. However it may compile just fine, I haven't personally tried. Feel free to give it a try and reply here if you encounter errors

[Sajjon]

@dapplion On a Mac Mini M1, using these version of yarn, node and npm (using NVM):

~/Developer/TypeScript/lodestar on  master ⌚ 8:11:19
$ node --version
v16.0.0

~/Developer/TypeScript/lodestar on  master ⌚ 8:14:07
$ npm --version
7.10.0

~/Developer/TypeScript/lodestar on  master ⌚ 8:14:12
$ yarn --version
1.22.10

Being on master branch (commit: 18f6cf5e860263c0e2e2cbe0aaf4f27761177855)

Running yarn test:unit

I get this error in @chainsafe/lodestar-cli test suite

@chainsafe/lodestar: $ nyc --cache-dir .nyc_output/.cache -e .ts mocha 'test/unit/**/*.test.ts'
@chainsafe/lodestar: dyld: lazy symbol binding failed: Symbol not found: __Z18sse42_is_availablev
@chainsafe/lodestar:   Referenced from: /Users/sajjode/Developer/TypeScript/lodestar/node_modules/sse4_crc32/build/Release/crc32c.node
@chainsafe/lodestar:   Expected in: flat namespace
@chainsafe/lodestar: dyld: Symbol not found: __Z18sse42_is_availablev
@chainsafe/lodestar:   Referenced from: /Users/sajjode/Developer/TypeScript/lodestar/node_modules/sse4_crc32/build/Release/crc32c.node
@chainsafe/lodestar:   Expected in: flat namespace
@chainsafe/lodestar: error Command failed with signal "SIGKILL".
@chainsafe/lodestar: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
@chainsafe/lodestar-cli: $ nyc --cache-dir .nyc_output/.cache -e .ts mocha 'test/unit/**/*.test.ts'
@chainsafe/lodestar-cli: dyld: lazy symbol binding failed: Symbol not found: __Z18sse42_is_availablev
@chainsafe/lodestar-cli:   Referenced from: /Users/sajjode/Developer/TypeScript/lodestar/node_modules/sse4_crc32/build/Release/crc32c.node
@chainsafe/lodestar-cli:   Expected in: flat namespace
@chainsafe/lodestar-cli: dyld: Symbol not found: __Z18sse42_is_availablev
@chainsafe/lodestar-cli:   Referenced from: /Users/sajjode/Developer/TypeScript/lodestar/node_modules/sse4_crc32/build/Release/crc32c.node
@chainsafe/lodestar-cli:   Expected in: flat namespace
@chainsafe/lodestar-cli: error Command failed with signal "SIGKILL".
@chainsafe/lodestar-cli: info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
lerna ERR! Received non-zero exit code 1 during execution
lerna success run Ran npm script 'test:unit' in 11 packages in 46.0s:
lerna success - @chainsafe/lodestar-beacon-state-transition
lerna success - @chainsafe/lodestar-cli
lerna success - @chainsafe/lodestar-config
lerna success - @chainsafe/lodestar-db
lerna success - @chainsafe/lodestar-fork-choice
lerna success - @chainsafe/lodestar-light-client
lerna success - @chainsafe/lodestar
lerna success - @chainsafe/lodestar-params
lerna success - @chainsafe/lodestar-types
lerna success - @chainsafe/lodestar-utils
lerna success - @chainsafe/lodestar-validator
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I think all other packages test suites passed though.

So seems it is the package sse4_crc32 that fails. Any ideas?

[Sajjon]

In error log above we see reference to something something "is_available", by searching for that in sse4 we find the function table_is_availabe and in code references to Intel. Which ofc will no be present on a non Intel machine :D.

Turns out it is @chainsafe/snappy-stream (repo kesla/node-snappy-stream) which depends on ashi009/node-fast-crc32c which has sse4_crc32 as an optionalDependencies.

Hmm can we remove/replace snappy-stream? Or update it to not rely on ashi009/node-fast-crc32c?

[dapplion]

@Sajjon Can you try to install the latest verion of fast-crc32c (2.0.0) in some empty node repo to check it compiles in M1 chips? Then I can bump the dependencies

[Sajjon]

@dapplion got same error:

$ node index.js
dyld: lazy symbol binding failed: Symbol not found: __Z18sse42_is_availablev
  Referenced from: ~/Developer/TypeScript/TestFastCRC32c/node_modules/sse4_crc32/build/Release/crc32c.node
  Expected in: flat namespace

dyld: Symbol not found: __Z18sse42_is_availablev
  Referenced from: ~/Developer/TypeScript/TestFastCRC32c/node_modules/sse4_crc32/build/Release/crc32c.node
  Expected in: flat namespace

[1]    17395 abort      node index.js

[dapplion]

@dapplion got same error:

$ node index.js
dyld: lazy symbol binding failed: Symbol not found: __Z18sse42_is_availablev
  Referenced from: ~/Developer/TypeScript/TestFastCRC32c/node_modules/sse4_crc32/build/Release/crc32c.node
  Expected in: flat namespace

dyld: Symbol not found: __Z18sse42_is_availablev
  Referenced from: ~/Developer/TypeScript/TestFastCRC32c/node_modules/sse4_crc32/build/Release/crc32c.node
  Expected in: flat namespace

[1]    17395 abort      node index.js

hmm that's a bummer. snappy stream is mandatory for the Eth2.0 protocol, we must use it. @mpetrunic can you take a look and see we have other options for that architecture?

[mpetrunic]

It's probably easier to refund M1 macbook for intel one than to force every software to support M1 (I think that's what I'm gonna do anyways) 🤣

We should thread carefully as it's something we use extensively (it's used to calculate and verify checksume of every snappy frame) and it could have significant performance impact.

We could replace fast-crc32c in snappy-stream with:

• https://www.npmjs.com/package/@node-lightning/checksum
  or
• https://www.npmjs.com/package/polycrc

[Sajjon]

@mpetrunic I just tried @node-lightning/checksum and it works like a charm on my M1 Mac Mini.

[dapplion]

@Sajjon Could you try to install in M1 Mac this two libraries? They are the others that may cause trouble too

• https://www.npmjs.com/package/bcrypto
• https://www.npmjs.com/package/@chainsafe/blst

[Sajjon]

@dapplion both packages worked 🎉 wrote tiny test code which executed beautifully 🚀