Incompatible with Macs based on the Apple M1 chip

[artob]

I'm recording this as a current known limitation: nearcore currently requires an x86 host which supports the AVX extensions.

nearcore/runtime/near-vm-runner/src/wasmer_runner.rs

Lines 191 to 196 in daad871

	if !cfg!(target_arch = "x86") && !cfg!(target_arch = "x86_64") {
	// TODO(#1940): Remove once NaN is standardized by the VM.
	panic!(
	"Execution of smart contracts is only supported for x86 and x86_64 CPU architectures."
	);
	}

Apple's Rosetta does not support AVX as of macOS 11.1 (Big Sur):

Rosetta translates all x86_64 instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions. If you include these newer instructions in your code, execute them only after verifying that they are available.

This effectively precludes running nearcore on Apple Silicon, such as the initial Apple M1 chip found in the new 2020 MacBook Pro, MacBook Air, and Mac Mini. While that's unlikely to be an issue for any production deployments, it will increasingly hamper our development as developers switch to new Macs.

The future is NEAR and the future is ARM (on Macs, at least), so we better figure out how to reconcile the two before too long.

[vgrichina]

While that's unlikely to be an issue for any production deployments

probably not this year, but given how much of energy savings M1 gives – I'm pretty sure data centers will start migrating once there is rack mounted option (i.e. likely within next 2 years)

[vgrichina]

check this mind blowing benchmark for neural nets training https://wandb.ai/vanpelt/m1-benchmark/reports/Can-Apple-s-M1-help-you-train-models-faster-cheaper-than-NVIDIA-s-V100---VmlldzozNTkyMzg

[artob]

Related pull request: #3799. Related upstream pull request: wasmerio/wasmer#1831

[ailisp]

@artob I recently worked with new version of wasmer, they support wasmer on ARM, this issue is on our side: when it's not x86, we should not require AVX

[ailisp]

And this one

// TODO(#1940): Remove once NaN is standardized by the VM. =

they claimed fixed: wasmerio/wasmer#1269

[ailisp]

@olonho pointed out that Linux ARM and Mac ARM is different compiler backend for Rust and also different rust crates availability, so although I can do some initial attempt on compile rust on a linux arm server, it doesn't mean it would work or not work on a M1 Macbook. @olonho recently have a M1 Macbook, so let me assign him to further investigate this issue :D

[ailisp]

FYI rust-rocksdb/rust-rocksdb#483 rust-rocksdb current release also have problem on new mac, waiting for this PR

[artob]

To aid in testing M1 support, note that Scaleway have just rolled out M1 hardware in the cloud, Apple-Silicon-as-a-Service:

https://www.scaleway.com/en/hello-m1/

[webmaster128]

While there is M1 support for Wasmer in general, there does not seem to be M1 support for Wasmer+Singlepass. This is documented in https://github.com/wasmerio/wasmer/blob/2.0.0/Makefile#L33-L37 and https://docs.wasmer.io/ecosystem/wasmer/wasmer-features#compiler-support-by-chipset.

I don't know if running x86_64 Singlepass on an M1 is feasible.

I guess this ticket is blocked by wasmerio/wasmer#2463.

[Kouprin]

check this mind blowing benchmark for neural nets training https://wandb.ai/vanpelt/m1-benchmark/reports/Can-Apple-s-M1-help-you-train-models-faster-cheaper-than-NVIDIA-s-V100---VmlldzozNTkyMzg

FYI https://www.reddit.com/r/MachineLearning/comments/kwqev4/d_comparing_performance_on_apples_m1_with_nvidias/

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[bowenwang1996]

Assigning to @matklad since he mentioned that he has a temporary solution

[matklad]

I believe rustup set default-host x86_64-apple-darwin makes nearcore to compile to x86 on M1, and that works via rosetta. At least, I've heard reports of that working, although I didn't try that myself.

The question of compiling near to ARM natively is more involved, for two reasons:

• we use a jit compiler, and so that should support ARM
• we need to estimate costs of operations, and today we use x86 as a model hardware

Could someone confirm/deny that setting host tripple to x86 works for modern macs?

[khorolets]

rustup set default-host x86_64-apple-darwin

Trying that now, will edit this message with results.

UPDATED: yep, this worked. Cold nearcore build on 13" M1 8 RAM is done in 6m 36s

[bowenwang1996]

UPDATED: yep, this worked. Cold nearcore build on 13" M1 8 RAM is done in 6m 36s

Now I regret not getting a M1 Macbook :(

[bowenwang1996]

@matklad could you how to compile nearcore on M1 to readme and close this issue? Thanks!

[ailisp]

This makes nearcore development possible on M1, which is great. Yet do we still want an issue to support "M1 native nearcore"?

…

On Mon, Oct 25, 2021 at 11:46 PM near-bulldozer[bot] < ***@***.***> wrote: Closed #3803 <#3803> via #5077 <#5077>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3803 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADFFFCB2C2ZB57TYJNYXVHLUIV3WRANCNFSM4WBOUGDQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[matklad]

Yet do we still want an issue to support "M1 native nearcore"?

I don't think so -- we rather fundamentally depend on a specific machine architecture for cost estimations. So, while in theory it should be possible to make nearcore node that runs on arm with correct costs, I feel that's a lot of work for relatively little benefit.

For developer convenience, "set default-host" seems like an OK solution, and we probably can make it better when we pull #3803.

It probably makes sense to revisit this in a couple of years, but we don't need an issue for that.

[khorolets]

rustup set default-host x86_64-apple-darwin

Trying that now, will edit this message with results.

UPDATED: yep, this worked. Cold nearcore build on 13" M1 8 RAM is done in 6m 36s

OK, my bad, at that moment I've just checked cargo build --release -p neard and the posted the result.

Recently, I've decided to use neard and failed.

I guess it was expected according to different architecture, but still I guess it's worth mentioning in this issue.

Running target/debug/neard init --chain-id localnet
fish: Job 1, 'cargo run -p neard -- init --...' terminated by signal SIGBUS (Misaligned address error)

/cc @matklad

[matklad]

I've had a call with dev-platform team and realised that we really need a seamless support for M1 (and, more generally, a seamless cross-platform story). The reason for that is sandbox -- the local instance of neard that contract developers to test their contracts.

[artob]

Recently, I've decided to use neard and failed.

I guess it was expected according to different architecture, but still I guess it's worth mentioning in this issue.

Building nearcore for x86 was not the issue, running it was. Runtime errors were to be expected, as per the original ticket description:

Apple's Rosetta does not support AVX as of macOS 11.1 (Big Sur):

Rosetta translates all x86_64 instructions, but it doesn’t support the execution of some newer instruction sets and processor features, such as AVX, AVX2, and AVX512 vector instructions. If you include these newer instructions in your code, execute them only after verifying that they are available.

[artob]

I've had a call with dev-platform team and realised that we really need a seamless support for M1 (and, more generally, a seamless cross-platform story). The reason for that is sandbox -- the local instance of neard that contract developers to test their contracts.

Good to hear, given that I, like increasingly many developers elsewhere, haven't used x86 for going on a year by now and don't expect that to change back.

[the-mercenaries-ltd]

to confirm this means m1's cannot run the local develop env described in: https://docs.near.org/docs/tools/kurtosis-localnet

or is there a workaround?

[rsilvestre]

Now I regret not getting a M1 Macbook :(

Now I regret getting a M1 Macbook Pro :(

[frol]

I have tested neard with #6200 patch and it works great on my Macbook with Apple M1 chip (except that I needed to hotfix #6293 as otherwise the node crashes on boot)!

While neard cannot be used on M1 in production given that it uses wasmtime instead of wasmer, I believe this particular issue should be closed as the main concern was usability during development cycles and it is now addressed! Kudos to @Ekleog 🎉

@Ekleog If you believe there is still some work left to be done, feel free to re-open this issue or open a new one.

[ChzenChzen]

@frol #6200 (comment)