Support for Ubuntu Linux on ARM64 (aarch64) #4119

jbeker · 2021-03-29T19:04:36Z

Problem

We are starting to need to use ARM based Linux (using Ubuntu) but we can't use Sorbet since there is not a binary version of Sorbet that will run. When performing a bundle install that includes sorbet we get the following error:

Unable to find a spec satisfying sorbet-static (= 0.5.5912) in the set. 
Perhaps the lockfile is corrupted? Found sorbet-static (0.5.5912-x86_64-linux), 
sorbet-static (0.5.5912-universal-darwin-19) that did not match the current platform.

This is running on Ubuntu 20 LTS

Linux local 5.4.0-70-generic #78-Ubuntu SMP Fri Mar 19 13:29:32 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

Proposed solution

Add a linux aarch64 pre-built sorbet-static.

The text was updated successfully, but these errors were encountered:

lox · 2021-06-09T09:54:13Z

Are there any workarounds for this in the meantime?

cguess · 2021-10-19T18:43:08Z

Has there been any updates on this? With the launch of even more M1 ARM-based Macs it's quickly becoming the soon-to-be-default architecture, and if you're running a VM there's no option except aarch64 for Linux (can't emulate x86).

korbin · 2021-10-22T22:52:34Z

Has there been any updates on this? With the launch of even more M1 ARM-based Macs it's quickly becoming the soon-to-be-default architecture, and if you're running a VM there's no option except aarch64 for Linux (can't emulate x86).

You could use QEMU to run Sorbet, probably.

I was able to make ARM64 builds work for Linux aarch64 targets:

master...angellist:arm64

I am not familiar enough with Bazel to add all of the conditionals necessary to make this properly support cross-platform builds, however, my findings:

Bazel upstream isn't publishing installers for other architectures now. I've opted to directly install platform Bazel binaries.
libb2 depends on SSE2 instructions. I removed it in favor of the existing blake2/blake2 implementation which seems to have optional SSE support.
Inline x86 assembly can be changed (safe to merge upstream now I assume): __asm__("int $3"); -> __builtin_debugtrap();

I also attempted to get Darwin ARM64 support working (and it probably will if these caveats are fixed), however:

The LLVM project does not publish Darwin ARM64 binaries. Homebrew clang or system clang will need to be used. (I am not familiar enough with Bazel yet to build this escape hatch cleanly.)
- Consequently, all builds will always become x86_64 due to the use of emulated x86_64 clang.
jemalloc needs to be downloaded from the dev branch.

Hopefully this helps enable upstream ARM support!

cguess · 2021-11-05T14:32:48Z

@korbin QEMU can emulate x64, but it is SLOW, pretty much unusable.

klardotsh · 2021-12-02T06:23:53Z

this is a blocker to teams using sorbet in docker as well - let me know if there's anything I can do to help move this forward; as it stands I believe my workaround has to be forcing platform: linux/amd64 for our containers, which is unideal.

EDIT: nope, that doesn't work because of the lack of inotify in qemu and the unsuitability of the "Experimental Virtualization Backend" for our purposes. I'll be working to try to build arm64 native sorbet for my team.

froydnj · 2021-12-03T14:12:48Z

let me know if there's anything I can do to help move this forward

Can you at least submit a PR for a7cdf0b ? I think the other commits would require some reworking (all the s/x86_64/aarch64/-style ones) or some testing (the blake2 changes), but the __builtin_debugtrap seems uncontroversial, assuming GCC supports it (i.e. we can build in Stripe's CI)

froydnj · 2021-12-03T14:29:24Z

I think we'd take a PR for the bazel script changes too!

klardotsh · 2021-12-03T23:38:50Z

A brief end of week update on my end: I haven't fully inspected the generated gems or run the test suite against them, but building on the work from AngelList above, I seem to have at least runnable aarch64 sorbet ELF binaries generated on an aarch64 host (read: not cross-compiled from amd64). The logic roughly being that usually I find on-host builds easier to reason about than cross-compiled, so I decided to head down the path I was most comfortable with in general (I've dealt with ARM Linux stuff for years, but almost never cross-compiled).

The commit logs are a little messy (especially on the bazel-toolchain side), there's decidedly some hackery going on that needs cleaned up because I don't particularly know Bazel well, and just broadly it'll need work before it'll be in a state to work with @korbin @froydnj and co. to upstream it, but it's progress and I'll likely continue poking at this Tuesday, if not over the weekend.

The below output is copied from an M1 Mac, but I was also able to use Docker BuildX to "cross-compile" (very, very, miserably slowly) from my AMD 3900X rig running Void Linux.

joshklar@twg-mba-klar sorbet % make release-linux-arm64                                                                                                                                                                                            
# this is PROBABLY dangerous in the event your system already has bin                                                                                                                                                                              
# handlers or something but whatever, those are *super* uncommon,                                                                                                                                                                                  
# especially on CI rigs                                                                                                                                                                                                                            
[ "$(uname -s) $(uname -m)" == "Linux x86_64" ] && docker run --rm --privileged multiarch/qemu-user-static --reset -p yes || true                                                                                                                  
docker buildx build --platform linux/arm64 .                                                                                                                                                                                                       
                                                                                                                                                                                                                                                   
[+] Building 516.1s (10/10) FINISHED                                                                                                                                                                                                               
 => [internal] load build definition from Dockerfile                                                                                                                                                                                          0.0s 
 => => transferring dockerfile: 573B                                                                                                                                                                                                          0.0s 
 => [internal] load .dockerignore                                                                                                                                                                                                             0.0s 
 => => transferring context: 2B                                                                                                                                                                                                               0.0s 
 => [internal] load metadata for docker.io/library/ubuntu:focal                                                                                                                                                                               1.1s 
 => [1/5] FROM docker.io/library/ubuntu:focal@sha256:626ffe58f6e7566e00254b638eb7e0f3b11d4da9675088f4781a50ae288f3322                                                                                                                         0.0s 
 => [internal] load build context                                                                                                                                                                                                             5.6s 
 => => transferring context: 195.39MB                                                                                                                                                                                                         5.5s 
 => CACHED [2/5] RUN apt update && apt install -y build-essential curl perl unzip python autoconf                                                                                                                                             0.0s 
 => CACHED [3/5] RUN apt update && apt install -y libffi-dev libgmp-dev libtinfo5 libtinfo-dev python python3                                                                                                                                 0.0s 
 => [4/5] COPY . .                                                                                                                                                                                                                            0.8s 
 => [5/5] RUN ./bazel build //main:sorbet --config=release-linux                                                                                                                                                                            491.5s 
 => exporting to image                                                                                                                                                                                                                       17.1s 
 => => exporting layers                                                                                                                                                                                                                      17.1s 
 => => writing image sha256:aa08645ee8b33bba601a9140a4d33faeb23fe2a71cc592d60e2eb0f03af19624                                                                                                                                                  0.0s 

# ...

joshklar@twg-mba-klar sorbet % docker run --rm -it aa08645ee8b33bba601a9140a4d33faeb23fe2a71cc592d60e2eb0f03af19624                                                                                                                                
root@478d1ec030c9:/# file ./root/.cache/bazel/_bazel_root/6666cd76f96956469e7be39d750cc7d9/execroot/com_stripe_ruby_typer/bazel-out/aarch64-opt/bin/main/sorbet                                                                                    
./root/.cache/bazel/_bazel_root/6666cd76f96956469e7be39d750cc7d9/execroot/com_stripe_ruby_typer/bazel-out/aarch64-opt/bin/main/sorbet: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[md5/uuid]=121e6e884edc997ac510ce7630cc2037, with debug_info, not stripped
root@478d1ec030c9:/# ./root/.cache/bazel/_bazel_root/6666cd76f96956469e7be39d750cc7d9/execroot/com_stripe_ruby_typer/bazel-out/aarch64-opt/bin/main/sorbet                                                                                         
You must pass either `-e` or at least one folder or ruby file.                                                                                                                                                                                     
                                                                                                                                                                                                                                                   
Typechecker for Ruby                                                                                                                                                                                                                               
Usage:                                                                                                                                                                                                                                             
  sorbet [OPTION...] <path 1> <path 2> ...                                                                                                                                                                                                         
                                                                                                                                                                                                                                                   
  -e, string     Parse an inline ruby string (default: "")                                                                                                                                                                                         
  -q, --quiet    Silence all non-critical errors                                                                                                                                                                                                   
  -v, --verbose  Verbosity level [0-3]                                                                                                                                                                                                             
  -h,            Show short help                                                                                                                                                                                                                   
      --help     Show long help                                                                                                                                                                                                                    
      --version  Show version                                                                                                                                                                                                                      
                                                                                                                                                                                                                                                   
root@478d1ec030c9:/# ./root/.cache/bazel/_bazel_root/6666cd76f96956469e7be39d750cc7d9/execroot/com_stripe_ruby_typer/bazel-out/aarch64-opt/bin/main/sorbet -e 'puts "yay"'                                                                         
No errors! Great job.

Currently, Sorbet uses blake2 for wasm builds and libb2 for other builds. Originally, this was because libb2 is heavily optimized but not portable. Since that time, however, libb2 has been extended with optimized builds for common architectures. That means that it's now possible to use the same library everywhere. x86/arm platforms still benefit from optimized vectorized implementations, and other platforms can still use the portable fallback. This change simplifies and slightly speeds up the build, while removing one of the blockers to Linux AArch64 support as described in sorbet#4119.

joaolrpaulo · 2022-02-10T14:36:58Z

👋 Any news about this topic?

Will there be a new release of the sorbet-static for aarch64 anytime soon?

vinistock · 2022-02-10T14:54:43Z

Our team is working on support for the M1 macs tracked in this milestone. Based on the investigations posted here, I believe the changes needed for M1 support overlap with most of what is needed for Linux ARM64 as well - other than some configuration bits.

The changes in this experimental branch are enough to compile Sorbet to ARM64 on an M1 machine for both debug and release mode. However, we still need to figure out some build failures for the custom Ruby build and emscripten.

Additionally, that branch uses the bazel-toolchain version that has support for the ARM64 clang toolchains, which is still in a PR and not yet merged upstream.

After all of the code changes are merged and the configuration API is finalized on bazel-toolchain, we should be able to start releasing the new binaries.

klardotsh · 2022-02-11T20:08:44Z

please feel free to holler if there's anything we (Dockwa) can do to help, even if it's testing a quick thing. we've been using the linux/amd64 and linux/arm64 gems+docker images generated in https://github.com/dockwa/sorbet/commits/josh/arm64-more for a bit now and, short a PEBKAC issue I introduced with versioning at one point, haven't had issues (mind you, we fall back to upstream gems on Darwin-bare-metal because I never figured out cross-compilation, with Bazel or - experimentally at one point - with Zig's cross-compile-as-a-first-class-citizen toolchain)

excited to see this getting upstream traction!

froydnj · 2022-02-11T20:16:35Z

Note that getting all the necessary changes in to compile on arm64 is one step (and an important one!), but not the complete picture: we also need to cross-compile the appropriate bits on Buildkite (AFAICT, Buildkite does not seem to have native arm64 machines/containers available).

klardotsh · 2022-02-11T20:21:05Z

Certainly native arm64 machines would be ideal, but if those fail, a (somewhat slow) option (that we're using in our fork) is to QEMU around the problem - Docker BuildX sets up a binfmt_misc handler to route cross-arch builds through QEMU wrappers, simulating a "native" arm64 build. From what I can tell, Rust's https://github.com/cross-rs/cross does this as well. Certainly not saying it's ideal (our arm64 images take about 5-10x longer than our amd64 images, building on an AMD 3900X with 24 threads), but it's a potential workaround if the cross-compile-directly story gets hairy. Happy to help there as well as time allows, if needed.

klardotsh · 2022-07-20T22:57:02Z

Chiming in here again, curious to see what the blockers here are, and what my org could do to help. At this point we're still running my fork from the winter, which means v0.5.9613 (the last version I could get to cleanly rebase), complete with all of our packaging layers to get there. Simplifying back to a simple "bundle install" would be great.

louiscb · 2022-08-17T11:40:42Z

@klardotsh can you publish your forked version that works with arm64?

klardotsh · 2022-08-18T03:39:23Z

@klardotsh can you publish your forked version that works with arm64?

Ah sorry, I realize I'd talked about the WIP above and never mentioned that all this work has been public all along... just without instructions available. Here goes!

master of https://github.com/dockwa/sorbet is at v0.5.9613 (this limits you to Tapioca 0.7.x if you use check-shims ¹ ²). This is a set of 12 commits over AngelList's fork above; essentially all are packaging and build system related.

In theory one should be able to apply this to bare-metal aarch64+glibc Linux; pulling the linux/arm64 OCI image (docker pull blahblah --platform linux/arm64 works I think maybe possibly, otherwise dig into the multiarch manifest) and dumping it to a tarball ³ should allow portability outside of containers.

To apply some duck-typing theory: this looks like a hack, walks like a hack, and quacks like a hack, so it's definitely a hack. With some caveats however (notably, that things like Tapioca RBI generation must be done in a consistent environment - for us that's linux/amd64 with RAILS_ENV=development - due to two Tapioca issues ⁴ ⁵), it's been working for us at Dockwa since Dec/Jan ish.

Dockerfile

# https://github.com/dockwa/sorbet
FROM ghcr.io/dockwa/sorbet:0.5.9613 AS sorbet-gems

# the bins generated in the buster image are confirmed so far to be
# forward-compatible, so rather than rebuilding a bullseye image, we'll just
# mix and match this particular component
#
# https://github.com/dockwa/docker-watchman
FROM ghcr.io/dockwa/watchman:4.9.0-debian-buster AS watchman

FROM ruby:2.7.5-bullseye

# ...

# this version is/was required for arm64 support due to
# https://github.com/rubygems/rubygems/pull/5092
# (https://github.com/rubygems/rubygems/issues/5088)
RUN gem install bundler:2.2.33

# this (ab)uses awk's conditional print mechanics to find sorbet-static's
# x86_64 gem lock, duplicate it to a new line, and then find-replace over the
# duplicated line, described more in https://unix.stackexchange.com/a/675838
#
# without this, cryptic bundler errors occur in arm64 builds (only): not only
# is a dependency outright missing, but because it's a transient dependency,
# weird offshoots of https://github.com/rubygems/rubygems/pull/5092 (that are
# yet unfixed even in 2.2.33, it appears) pop up, all instances of the famous
# NilClass exception
RUN awk '1; gsub(/x86_64/,"aarch64")' Gemfile.lock > Gemfile.lock.new
RUN mv Gemfile.lock.new Gemfile.lock

# ... and since those issues stemmed from upstream not providing aarch64 bins
# in the first place, we'll provide our own *.gem for all things sorbet, and
# lean on bundler's implicit search path behaviors to avoid looking for this
# upstream (it won't exist)
COPY --from=sorbet-gems /sorbet*.gem ./vendor/cache/

RUN bundle install

# prep watchman (required for sorbet's LSP mode, which is sometimes called in
# this image via docker wrapper scripts hooked up to editors like neovim)
USER root
COPY --from=watchman /usr/local/bin/watchman* /usr/local/bin/
COPY --from=watchman /usr/local/share/doc/watchman-4.9.0 /usr/local/share/doc/watchman-4.9.0
COPY --from=watchman /usr/local/var/run/watchman /usr/local/var/run/watchman
RUN mkdir -p /usr/local/var/run/watchman
RUN touch /usr/local/var/run/watchman/.not-empty \
 && chown -R $(id -u app):$(id -g app) /usr/local/var/run/watchman
USER app

Makefile to wrangle BuildX

.POSIX:

REGISTRY_BASE ?= REDACTED
REGISTRY_BASE_IMAGE ?= $(shell git describe --always --abbrev)

ifeq ($(PUSH),1)
	BASE_PUSH_FLAG := --push
endif

ifeq ($(IS_TRUNK_CI_BUILD),1)
	BASE_LATEST_TAG := -t $(REGISTRY_BASE):latest
endif

.PHONY: multiarch-qemu-reset
multiarch-qemu-reset:
	# this is PROBABLY dangerous in the event your system already has bin
	# handlers or something but whatever, those are *super* uncommon normally
	#
	# DO NOT run this on GitHub Actions, use docker/setup-qemu-action@v1
	# instead!
	[ "$$(uname -s)" = "Linux" ] && docker run --rm --privileged multiarch/qemu-user-static --reset -p yes || true

.PHONY: buildx-base-image
buildx-base-image:
	docker buildx build \
		--platform linux/amd64,linux/arm64 \
		-t $(REGISTRY_BASE):$(REGISTRY_BASE_IMAGE) \
		$(BASE_PUSH_FLAG) \
		$(BASE_LATEST_TAG) \
		.

https://github.com/Shopify/tapioca/commit/03266889ca71a1114cd91a3cc9c3a0dd435c51d2 ↩
I've tried rebasing this branch onto something newer and Tapioca 0.8.x compatible and while the diffs eventually apply cleanly, I get Bazel-induced build failures. Bazel is a monstrously complex beast I have thus far consistently failed to wrap my head around (especially debugging it), and thus I gave up for now - I have too many non-Sorbet responsibilities at work to dedicate the requisite time to this. ↩
docker docs, podman docs ↩
https://github.com/Shopify/tapioca/issues/658 ↩
https://github.com/Shopify/tapioca/issues/657 ↩

klardotsh · 2022-09-09T21:47:30Z

I should, however, explicitly note now: my org is removing Sorbet from our codebase in the near future; I will likely no longer be maintaining the above forks or Docker images any further, and in particular, will not be continuing the efforts to rebase onto newer Sorbet releases. I have no intentions quite yet of removing the repos, and of course the contributions are all under the same license as Sorbet upstream, so feel free to make your own forks or otherwise remix my work so far ¹ :) Cheers y'all!

I'll see what I can do about open-sourcing some of our internal scripts and helper tooling around this, perhaps in a GitHub Gist or something, but no promises. ↩

ethankhall · 2022-09-27T16:58:34Z

For those interested, I took @klardotsh's changes and made them compatible with main (as of Sep 26, 2022) and able to build a static sorbet for linux-arm. You can find the changes here https://gist.github.com/ethankhall/516ddcefd22297a6cb9f736648386292.

technicalpickles · 2022-10-20T15:01:40Z

@bhuga this is the ARM64 issue I was telling you about

Nealsoni00 · 2023-05-30T13:18:50Z

Is there any expectation that ARM Linux will be supported in the future?

The issue makes Sorbet inoperable for a small team like mine developing on ARM Macs with everything unified into docker 😞. Our CI is also running using ARM and has the same challenge. I would LOVE to add the benefit of sorbet's type system!

v2kovac · 2023-06-04T06:19:02Z

Is there any expectation that ARM Linux will be supported in the future?

The issue makes Sorbet inoperable for a small team like mine developing on ARM Macs with everything unified into docker 😞. Our CI is also running using ARM and has the same challenge. I would LOVE to add the benefit of sorbet's type system!

@Nealsoni00 our team got around this by having a separate gemfile for your local machine and docker, you only need sorbet locally.
eg:

# local gemfile dont add stuff here
eval_gemfile './Gemfile.docker'
gem 'sorbet'
gem 'tapioca'

And then you can use the env variable BUNDLE_GEMFILE to specify which to use where. After this one time setup we've had no problems, been doing this since december.

nalgenewaterbottle · 2023-10-17T14:31:34Z

What can we do to help set up the ARM64 Buildkite agent?

mlarraz · 2023-10-18T15:23:54Z

What can we do to help set up the ARM64 Buildkite agent?

I believe this has to be done by somebody at Stripe. FYI @jez

tonybruess · 2023-10-18T16:09:31Z

Two months ago the last comment on this subject was from @froydnj in Slack:

we don’t use sorbet-build-image for builds inside Stripe

Sorbet releases on linux-arm64 (and darwin-arm64) haven’t been a priority for us at the moment

Until this is prioritized internally at Stripe it feels like we shouldn't expect to see arm64 releases. We may want to consider setting up a community run agent and repo.

froydnj · 2023-10-18T16:18:25Z

We (well, @jez) has been trying to set up a buildkite agent for arm64 builds, but the current status is that the build is getting peculiar errors while setting up the required Ubuntu PPA repos (it looks like it's trying to fetch from an entirely different PPA than the one specified in the Dockerfile). Things are moving, just maybe not as quickly as one might like.

noizwaves · 2023-10-18T16:41:53Z

Many thanks for this @froydnj & @jez !

it looks like it's trying to fetch from an entirely different PPA than the one specified in the Dockerfile

Do y'all have any build logs that you can share? Because of the work on sorbet/sorbet-build-image#8 I may be able to help.

froydnj · 2023-10-18T16:45:01Z

This is the end of the log:

#8 53.15 Cannot add PPA: 'ppa:~ubuntu-toolchain-r/ubuntu/test'.
#8 53.15 The team named '~ubuntu-toolchain-r' has no PPA named 'ubuntu/test'
#8 53.15 Please choose from the following available PPAs:
#8 53.15  * 'aarch64':  AArch64
#8 53.15  * 'binutils':  binutils
#8 53.15  * 'dpkg-lto':  dpkg defaulting to LTO flags
#8 53.15  * 'glibc':  glibc test builds
#8 53.15  * 'golang-fips':  Go FIPS
#8 53.15  * 'lunar-o3':  lunar O3
#8 53.15  * 'lunar-o3-v3':  lunar O3 v3
#8 53.15  * 'lunar-v3':  lunar v3
#8 53.15  * 'p9':  POWER9
#8 53.15  * 'power8':  POWER8
#8 53.15  * 'ppa':  Toolchain builds
#8 53.15  * 'security':  Security Uploads
#8 53.15  * 'test':  Toolchain test builds
#8 53.15  * 'volatile':  Volatile Packages
#8 53.15  * 'x86-64-v2':  GCC x86-64-v2
#8 53.15  * 'x86-64-v3':  GCC x86-64-v3

tonybruess · 2023-10-18T19:56:05Z

Thanks for prioritizing this @jez and @froydnj! We all greatly appreciate it.

That's a strange error. Where is the docker build being run?

noizwaves · 2023-10-23T02:48:41Z

Thanks for the extra info @froydnj & @jez. We really appreciate the effort here!

The latest code from GitHub builds for me on an M1 laptop with docker build --no-cache., so I suspect something environmental is contributing. I have a couple of theories/suggestions:

It sounds like add-apt-repository may have trouble in some HTTP(S)_PROXY setups, and reports the error in a general way. Is the image being built on a machine where a proxy is being used? If so, try passing the proxy env var directly to the docker build command like this: docker build --build-arg HTTP_PROXY="$HTTP_PROXY" ..

If your environment uses an HTTPS_PROXY, use docker build --build-arg HTTP_PROXY="$HTTP_PROXY" . instead.
If you're not secure what proxies are present, env | grep _PROXY will show them.

Replace the apt-add-repository command that achieve a similar (see patch below). Even if this doesn't work, it should yield a different error message 🤞 .

diff --git a/Dockerfile b/Dockerfile
index 1a79916..4de9908 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -17,7 +17,9 @@ RUN apt-get update && \
       echo "deb http://apt.llvm.org/bionic/ llvm-toolchain-bionic-9 main" | tee /etc/apt/sources.list.d/llvm.list && \
       apt-get update && \
       apt-get install --no-install-recommends -y nodejs yarn clang-9 && \
-      add-apt-repository --yes ppa:ubuntu-toolchain-r/test && \
+      apt-key adv --no-tty --keyserver keyserver.ubuntu.com --recv-keys 1E9377A2BA9EF27F && \
+      echo "deb http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu bionic main" | tee /etc/apt/sources.list.d/ubuntu-toolchain-r-ubuntu-test-bionic.list && \
+      echo "# deb-src http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu bionic main" | tee -a /etc/apt/sources.list.d/ubuntu-toolchain-r-ubuntu-test-bionic.list && \
       apt-get update && \
       apt-get install --yes --only-upgrade libstdc++6 && \
       cd bazel_loader && \

eric-christian · 2023-10-26T12:20:58Z

Hello,

Will this solve the problems for people on OSX with M1 chip, too? I'm able to install sorbet locally with gem install sorbet but it fails within a rails project when using bundler. So i'm a little confused about (in)compatibility.

❯ uname -a
Darwin mymacbookpro 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:34 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T8103 arm64

❯ bundle -v
Bundler version 2.4.19

❯ rails -v
Rails 7.1.1

❯ ruby -v
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]

❯ bundle install
Fetching gem metadata from https://rubygems.org/.........
Resolving dependencies...
Resolving dependencies...
Could not find gem 'sorbet-static' with platforms 'aarch64-linux', 'ruby', 'x86_64-linux' in rubygems repository https://rubygems.org/ or installed locally.

tonybruess · 2023-10-26T15:57:10Z

@eric-christian What does bundle config say? I've seen this issue if force_ruby_platform: true

eric-christian · 2023-10-27T07:57:16Z

@eric-christian What does bundle config say? I've seen this issue if force_ruby_platform: true

Yes, that was indeed on. Without, it works. Thank you!

Wonder why i had that on. Has probably something to do with a rails app i work on that requires all gems for all supported platforms to be packed within the vendor/cache. 🤔

nalgenewaterbottle · 2023-11-08T16:54:21Z

This sounds close! Thank you again to those that are working on this 😊

noizwaves · 2023-11-17T19:58:47Z

Hey @froydnj & @jez - hope you are both doing well. I just wanted to check in about this Docker image build issues you were experiencing, and if there were any new blockers?

I have some bandwidth to work on this at Gusto, and would love to help out where I can.

froydnj · 2023-12-01T14:58:02Z

Hey @froydnj & @jez - hope you are both doing well. I just wanted to check in about this Docker image build issues you were experiencing, and if there were any new blockers?

I have some bandwidth to work on this at Gusto, and would love to help out where I can.

Thanks for checking in @noizwaves ! Whatever issue we were experiencing seemed to be transient; I couldn't reproduce and jez couldn't reproduce it a day or so later.

I think the part we got stuck on was tagging the resulting images on Docker Hub. Our current practice right now is tagging the x86-64 image as latest, but that...probably won't work to tag both the x86-64 and arm64 images as latest? So we have to have different tags for each one, or is there magic that ensures we can have multiple architectures tagged as latest? (None of us are Docker experts in this respect.)

JamieMagee · 2023-12-01T15:19:09Z

You can make the latest tag into a manifest list. That allows Docker to pull the correct architecture for the platform it's on without you having to specify.

You'll need to create, tag, and push two architecture specific images, probably latest-amd64 and latest-arm64 but you can call them whatever you'd like. Then you can use the docker manifest command to create and push the manifest list. Something like:

docker manifest create sorbet:latest \
  --amend sorbet:latest-amd64 \
  --amend sorbet:latest-arm64
docker manifest push sorbet:latest

simoleone · 2023-12-01T15:31:23Z

There's a way to use the same tag for both architectures and many images out there do, but it's a bit of a pain to set up and troubleshoot to put it mildly.

A few random tips having dealt with this recently:

docker push/pull almost never do what you want when building/tagging/managing multi-platform images. They'll only deal with the platform they happen to be running on, and worse, do it silently. They are fine for running the images, of course, in which case they do what you want.
docker buildx --platform .... --push can take a comma-separated list of platforms. When you do this it will join them together into a single manifest list automatically and push them straight away. On the other hand, buildx can be horrendously slow to cross-build.
skopeo is a useful tool if you need to multi-platform "promote" images once they're already pushed, or push them around between repos, etc.

noizwaves · 2023-12-01T19:48:05Z

I'd second @JamieMagee 's approach. Failing that, it's also possible to skip the multi-platform tag and just use the platform specific tags. This does require some parametrization of the build pipeline, but that can be easier to get working.

noizwaves · 2023-12-01T19:50:39Z

The docs on docker manifest create include a very relevant example too.

nalgenewaterbottle · 2023-12-08T02:17:54Z

I would love to help as needed too! 😄 What are the relevant roadblocks?

noizwaves · 2023-12-09T17:35:22Z

I built and released an aarch64-linux and arm64-darwin-22 version if anyone wants to take it for a test run.

Hey @tonybruess 👋 , do you have the build process you used for the arm64-darwin-22 gem build anywhere? I have a little bandwidth and would like to build some linux arm64 gems for the community, and would love to expand this to darwin too.

tonybruess · 2023-12-11T03:58:28Z

@noizwaves Unfortunately I didn't save the exact steps anywhere. The only note I left for myself is:

Change /usr/local/var/bazelcache to /Users/name/.cache/bazel in .bazelrc.local

My org just decided to remove Sorbet entirely so unfortunately I don't have any bandwidth to support further. :(

noizwaves · 2023-12-11T16:46:12Z

Oh no, that sucks @tonybruess! Thank you for all the help so far on this issue ❤️

noizwaves · 2023-12-11T16:49:29Z

I have a little bandwidth and would like to build some linux arm64 gems for the community, and would love to expand this to darwin too.

Ok folks, I was able to get some time this weekend and have set up https://github.com/sorbet-multiarch. It builds the linux/arm64 Sorbet gem on CircleCI and publishes it to Gemfury. It automatically runs nightly, and should provide up to date linux/arm64 gems moving forward.

Currently version 0.5.11150 and 0.5.10993 are available. If you need another prior version, just create an issue over here and I'll build it for you.

nalgenewaterbottle · 2024-01-04T16:33:14Z

I'm curious if this is going to get official traction. Even with community solutions published and showing a near complete path for implementation, it just doesn't seem like there's room for it. Should we expect to use @noizwaves's implementation ? Which is great, and thank you for putting it forward.

Dandush03 · 2024-01-12T23:30:44Z

I have a little bandwidth and would like to build some linux arm64 gems for the community, and would love to expand this to darwin too.

Ok folks, I was able to get some time this weekend and have set up https://github.com/sorbet-multiarch. It builds the linux/arm64 Sorbet gem on CircleCI and publishes it to Gemfury. It automatically runs nightly, and should provide up to date linux/arm64 gems moving forward.

Currently version 0.5.11150 and 0.5.10993 are available. If you need another prior version, just create an issue over here and I'll build it for you.

GREAT WORK!

X-sam · 2024-02-16T17:42:15Z

does the announcement of new m1 runners for open source projects make this an easier ask?

nalgenewaterbottle · 2024-02-16T17:47:35Z

I don't think so @X-sam - I think the solutions provided above (from @noizwaves) actually accomplish the asks. It would seem it's more so on the time @jez / @froydnj and team have to maintain and implement it. Something that doesn't seem to be prioritized. But who knows! Maybe that was really the blocker.

jbeker added enhancement New feature or surprising current feature unconfirmed This issue has not yet been confirmed by the Sorbet team labels Mar 29, 2021

froydnj removed the unconfirmed This issue has not yet been confirmed by the Sorbet team label Feb 11, 2022

OAuthBringer mentioned this issue Mar 18, 2022

improvement: Consolidate and Update Readme, Add convenience tasks for Makefile. brave-intl/publishers#3577

Merged

fcheung mentioned this issue Sep 30, 2022

Could not find a valid gem 'sorbet-static' (= 0.5.10456) (required by 'sorbet' (>= 0)) in any repository #6429

Closed

ethankhall mentioned this issue Oct 18, 2022

Allow ARM Linux builds #6485

Closed

marcotc mentioned this issue Nov 29, 2022

Remove ruby platform check for sorbet dependency DataDog/dd-trace-rb#2445

Closed

jez mentioned this issue May 8, 2023

Could not find gem 'sorbet-static' with platforms 'aarch64-linux', #6985

Closed

noizwaves mentioned this issue Dec 11, 2023

sorbet-static-0.5.10667-universal-darwin-21 does not include arm64 code #6778

Open

DazWorrall mentioned this issue Feb 23, 2024

Build and release on linux-aarch64 #7718

Merged

jez closed this as completed in #7718 Feb 23, 2024

Support for Ubuntu Linux on ARM64 (aarch64) #4119

Support for Ubuntu Linux on ARM64 (aarch64) #4119

Comments

jbeker commented Mar 29, 2021 • edited Loading

Problem

Proposed solution

lox commented Jun 9, 2021

cguess commented Oct 19, 2021

korbin commented Oct 22, 2021 • edited Loading

cguess commented Nov 5, 2021

klardotsh commented Dec 2, 2021 • edited Loading

froydnj commented Dec 3, 2021

froydnj commented Dec 3, 2021

klardotsh commented Dec 3, 2021 • edited Loading

joaolrpaulo commented Feb 10, 2022

vinistock commented Feb 10, 2022

klardotsh commented Feb 11, 2022

froydnj commented Feb 11, 2022

klardotsh commented Feb 11, 2022 • edited Loading

klardotsh commented Jul 20, 2022

louiscb commented Aug 17, 2022

klardotsh commented Aug 18, 2022

Footnotes

klardotsh commented Sep 9, 2022 • edited Loading

Footnotes

ethankhall commented Sep 27, 2022

technicalpickles commented Oct 20, 2022

Nealsoni00 commented May 30, 2023 • edited Loading

v2kovac commented Jun 4, 2023

nalgenewaterbottle commented Oct 17, 2023

mlarraz commented Oct 18, 2023 • edited Loading

tonybruess commented Oct 18, 2023 • edited Loading

froydnj commented Oct 18, 2023

noizwaves commented Oct 18, 2023

froydnj commented Oct 18, 2023

tonybruess commented Oct 18, 2023

noizwaves commented Oct 23, 2023

eric-christian commented Oct 26, 2023 • edited Loading

tonybruess commented Oct 26, 2023

eric-christian commented Oct 27, 2023

nalgenewaterbottle commented Nov 8, 2023

noizwaves commented Nov 17, 2023

froydnj commented Dec 1, 2023

JamieMagee commented Dec 1, 2023 • edited Loading

simoleone commented Dec 1, 2023

noizwaves commented Dec 1, 2023

noizwaves commented Dec 1, 2023

nalgenewaterbottle commented Dec 8, 2023

noizwaves commented Dec 9, 2023

tonybruess commented Dec 11, 2023

noizwaves commented Dec 11, 2023

noizwaves commented Dec 11, 2023

nalgenewaterbottle commented Jan 4, 2024

Dandush03 commented Jan 12, 2024

X-sam commented Feb 16, 2024

nalgenewaterbottle commented Feb 16, 2024

jbeker commented Mar 29, 2021 •

edited

Loading

korbin commented Oct 22, 2021 •

edited

Loading

klardotsh commented Dec 2, 2021 •

edited

Loading

klardotsh commented Dec 3, 2021 •

edited

Loading

klardotsh commented Feb 11, 2022 •

edited

Loading

klardotsh commented Sep 9, 2022 •

edited

Loading

Nealsoni00 commented May 30, 2023 •

edited

Loading

mlarraz commented Oct 18, 2023 •

edited

Loading

tonybruess commented Oct 18, 2023 •

edited

Loading

eric-christian commented Oct 26, 2023 •

edited

Loading

JamieMagee commented Dec 1, 2023 •

edited

Loading