Boot hangs on M1 macs

[lycurgus]

Describe the bug
Startup hangs when urbit is run on an M1-based Apple machine.

To Reproduce
Steps to reproduce the behaviour:

1. Run urbit -w sampel-palnet -k /path/to/keyfile on an M1 mac
2. Note that the output stops after:

~
urbit 1.0
boot: home is /Users/username/urbit/sampel-palnet

Expected behaviour
Boot should proceed normally.

Screenshots
If applicable, add screenshots to help explain your problem.

System (please supply the following information, if relevant):

• OS: macOS
• Vere: v1.0
• Your ship's %base hash (use +trouble to check): n/a, unable to complete boot process

Additional context
Reporting this on behalf of a couple of users on Discord who have encountered this, so my ability to test things is going to be limited.

[zodcreator]

Yep, same problem here.

[joemfb]

M1 macs will require their own release binaries, right now we're only building for x86-64.

If any M1 pilots are game, I'd be curious to know if our current build works locally without changes.

/cc @brendanhay

[ohAitch]

Curious if anyone has tried running it under arch -x86_64

…

On Mon, Jan 11, 2021 at 09:49, Joe Bryan ***@***.***> wrote: M1 macs will require their own release binaries, right now we're only building for x86-64. If any M1 pilots are game, I'd be curious to know if our current build works locally without changes. /cc @brendanhay <https://github.com/brendanhay> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4257 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOFPBVSBQTTQ2TQY6H7GH3SZM23TANCNFSM4V33MHSQ> .

[louisbirla]

I use an M1 mac and ./urbit -c mycomet (from the install page) hung. arch -x86_64 ./urbit -c mycomet also hung.

[botter-nidnul]

If any M1 pilots are game, I'd be curious to know if our current build works locally without changes.

No easy way to build it yet, the aarch64-darwin version of nix is still extremely experimental.

[galenwp]

Looks like it's coming along: NixOS/nixpkgs#95903 (comment)

[botter-nidnul]

Looks like things have matured enough, trying to build it shouldn't be a herculean effort. If somebody wants to follow the steps in this comment and the comment above it, you should be able to get an aarch64-darwin urbit.

I'd recommend just trying to build the normal nix, dynamic c binaries. Don't try static versions or urbit-king unless you want extra challenge, and trying to build urbit-king probably just won't work (but, if it did, also might cause a not insignificant amount of ssd wear with the abnormally high swap usage of the m1)

There might be problems with the lmdb mapsize with macos on aarch64, just like I discovered with linux on aarch64. Have to see if it runs without modification.

[locpyl-tidnyd]

If nix support on M1 is still too flaky, one could reuse the poor-mans-nix-shell.sh approach in mingw-port.

[locpyl-tidnyd]

Commit 23634f6 in #4675 ought to do the trick. This build process uses an updated poor-mans-nix-shell.sh, Apple clang compiler and Homebrew. Instructions:

1. Install Homebrew in default location (/opt/homebrew). This should also install Xcode command line tools (CLT). Enable developer mode with sudo DevToolsSecurity -enable. You may have to add Homebrew's bin directory to PATH manually; not sure how that's supposed to work.
2. Download and unpack the latest mingw-port tarball.
3. Go to pkg/urbit and run HOST=aarch64-m1brew-apple ./configure. This will install the required Homebrew packages, prompt you to restart in case your bash is too old, download and build urbit dependencies.
4. Run make build/urbit build/urbit-worker. The resulting urbit binaries should be redistributable. They link dynamically only to OSX-provided libraries (OSX SDK and zlib). All other dependencies are linked statically. You can verify this with otool -L build/urbit.

Incidentally, I found out why boot was hanging: on aarch64-apple-darwin, the location loom was mmaped to happened to be right in the middle of every shared library's writable data, including the dynamic linker. mmap was instantly overwriting all this data with zeros, resulting in some truly bizarre behavior. I've moved the loom much higher in address space to avoid this. Other than this, I did not have to make any fixes in urbit sources. I was able to boot a comet, use Dojo from the terminal, open Landscape, and |hi other ships with the resulting binaries.

[galenwp]

This sounds very promising

[liam-fitzgerald]

Can confirm that changing the loombase as per 23634f6, compiling a static build on a intel mac will result in a working binary on apple silicon. Will be running under rosetta, but it's a start.

[mattlevan]

Commit 23634f6 in #4675 ought to do the trick. This build process uses an updated poor-mans-nix-shell.sh, Apple clang compiler and Homebrew. Instructions:

1. Install Homebrew in default location (/opt/homebrew). This should also install Xcode command line tools (CLT). Enable developer mode with sudo DevToolsSecurity -enable. You may have to add Homebrew's bin directory to PATH manually; not sure how that's supposed to work.
2. Download and unpack the latest mingw-port tarball.
3. Go to pkg/urbit and run HOST=aarch64-m1brew-apple ./configure. This will install the required Homebrew packages, prompt you to restart in case your bash is too old, download and build urbit dependencies.
4. Run make build/urbit build/urbit-worker. The resulting urbit binaries should be redistributable. They link dynamically only to OSX-provided libraries (OSX SDK and zlib). All other dependencies are linked statically. You can verify this with otool -L build/urbit.

Incidentally, I found out why boot was hanging: on aarch64-apple-darwin, the location loom was mmaped to happened to be right in the middle of every shared library's writable data, including the dynamic linker. mmap was instantly overwriting all this data with zeros, resulting in some truly bizarre behavior. I've moved the loom much higher in address space to avoid this. Other than this, I did not have to make any fixes in urbit sources. I was able to boot a comet, use Dojo from the terminal, open Landscape, and |hi other ships with the resulting binaries.

This works for me on my M1, just make sure to use the 23634f6. I just checked out this commit from the jb/urcrypt-prep branch in the main urbit repo and ran the configure and make commands from there. Beware you may run into "duplicate symbols" errors (from ld) if you have multiple openssl packages installed via brew (i.e., one in the main brew bin and one in the Cellar). In my case, I just uninstalled the couple of packages that depended on the Cellar version.

[locpyl-tidnyd]

GitHub Actions say they plan to have M1 runners up by EOY. It's probably best to put off setting up official M1 builds till then, as setting up an M1 build server and hooking it up as an external runner is a bit hairy, and on the other hand x64 Urbit binaries run fine with Rosetta.

[ghost]

building and running on m1 seems to work now

[liam-fitzgerald]

It does indeed, closing @joemfb

[jpzk]

when release binaries for m1?

[matthew-levan]

when release binaries for m1?

https://github.com/urbit/vere/releases check out the macos-aarch64.tgz file in the latest release.