Support major Linux distros on arm64 hardware

[znmeb]

I'm in the process of porting the open source version of RStudio Server 1.4 to NVIDIA Jetsons, which are arm64 CPUs with attached NVIDIA GPUs. The OS is Ubuntu 18.04 LTS "Bionic Beaver" with some extra repositories for the NVIDIA packages.

What I've discovered is that there are a few places where the build breaks because arm64 binaries aren't readily available. The most glaring example is pandoc. I'm building that from source, which is an overnight process on an eight-core developer kit with 16 GB of RAM and won't even run on a four-core 4 GB developer kit.

I have the steps worked out for Ubuntu 18.04; I haven't tried it on other distros but the process is fully Dockerized and should work with any distro that has an arm64 base image. It does not require any NVIDIA hardware or software; that's just what I have available. And I haven't been able to find a CI/CD platform with the hardware required to automate all of this.

[ronblum]

@znmeb Thank you for the request! I'll mark it for review as part of our ongoing development of RStudio Server.

[jmcphers]

See also adjacent issue #8652 which tracks adding arm64 support for Apple silicon.

[znmeb]

See also adjacent issue #8652 which tracks adding arm64 support for Apple silicon.

Is the new Apple silicon significantly more powerful than an equivalent priced x86-64? My Xavier has 8 2.26 GHz cores and the pandoc build takes several hours, mostly single-threaded. If I could afford it I'd move to an AWS Graviton CI/CD platform.

If you want to look over my shoulder, the hacks to the install scripts are all here: https://github.com/edgyR/edgyR-containers/tree/rstudio-1.4/build-scripts/internal-ubuntu-builder/dependencies. That will be moving into the main branch in about a week.

[elbamos]

@znmeb This is off-topic, but you may encounter issues with GHC compiling pandoc on arm64. See Homebrew/homebrew-core#65997 (comment) for discussion.

And to answer your question, yes, the Apple silicon is significantly more powerful than x86. On my cheap laptop, x86 applications run as fast under emulation as they do on a state-of-the-art laptop from six months ago. For native applications, the jump in performance reminds me of the jumps we got used to getting every 18 months in the 90s. The jump is larger for things like Apple's build of tensorflow that have been optimized for the chips.

With R, benchmarks are dramatically improved for everything except some matrix operations, which I assume is because they don't take advantage of a natively-optimized BLAS.

[znmeb]

@znmeb This is off-topic, but you may encounter issues with GHC compiling pandoc on arm64. See Homebrew/homebrew-core#65997 (comment) for discussion.

It's not off-topic as long as the Pandoc team doesn't provide binaries for arm64 and we're talking about porting RStudio. :-)

I'll take a look at that - the GHC in Bionic is 8.0; I tried a few more recent ones and they were better but they take time to recompile themselves.

And to answer your question, yes, the Apple silicon is significantly more powerful than x86. On my cheap laptop, x86 applications run as fast under emulation as they do on a state-of-the-art laptop from six months ago. For native applications, the jump in performance reminds me of the jumps we got used to getting every 18 months in the 90s. The jump is larger for things like Apple's build of tensorflow that have been optimized for the chips.

With R, benchmarks are dramatically improved for everything except some matrix operations, which I assume is because they don't take advantage of a natively-optimized BLAS.

Hmmm ... I'm still using the BLAS in R source. At some point I'll get around to experimenting with other ones.

[elbamos]

@znmeb The BLAS issue is that the M1 processor has "tensor cores" for accelerating matrix operations, as well as its own extended instruction set for vectorized operations. I believe the R built-in BLAS includes optimizations that take advantage of the x86 extended instruction set, and in any case it was easily replaced by native BLAS that did.

[znmeb]

@znmeb The BLAS issue is that the M1 processor has "tensor cores" for accelerating matrix operations, as well as its own extended instruction set for vectorized operations. I believe the R built-in BLAS includes optimizations that take advantage of the x86 extended instruction set, and in any case it was easily replaced by native BLAS that did.

I'd have to go look at the code but my recollection is that the optimizations for R's included BLAS were done with the compiler, which for Fortran is probably gcc. It does sound like it's worth picking up a Mac Mini once it's supported by Ubuntu for dual-booting.

[elbamos]

@znmeb Have you made any progress? A (perhaps dockerized) rstudio server build for arm would be a good interim step for those of us with M1s. I looked through your repositories but couldn't find the Dockerfile to build rstudio server on arm.

[znmeb]

Sorry - I got caught up in a new project. I'm going to push out the release documentation as soon as possible, but here are the Docker contexts:

1. Pandoc 2.11.2: https://github.com/edgyR/edgyR-containers/tree/main/build-scripts/internal-pandoc. This runs off the Ubuntu arm64 image, i.e., no NVIDIA dependencies. It takes a long time to build and uses a lot of RAM.
2. R, RStudio Server 1.4, and the libnode-dev package that the R V8 package requires: https://github.com/edgyR/edgyR-containers/tree/main/build-scripts/internal-ubuntu-builder. This builds on the first image and also has no NVIDIA dependencies.
3. The edgyR image: https://github.com/edgyR/edgyR-containers/tree/main/build-scripts/edgyr. This is the end-user image, which only runs on an NVIDIA Jetson. There are some things on there you might want on an Apple Silicon Mac, like MiniForge (https://github.com/conda-forge/miniforge) and Julia.

[elbamos]

I built a working docker based on @znmeb 's work and the Rocker scripts. I was able to get the size down to 1.8GB, which is still too big, but workable I guess: https://hub.docker.com/r/amoselb/rstudio-m1

[znmeb]

I built a working docker based on @znmeb 's work and the Rocker scripts. I was able to get the size down to 1.8GB, which is still too big, but workable I guess: https://hub.docker.com/r/amoselb/rstudio-m1

For what platform? And how does your Docker build differ from what ships with the RStudio source tree?

The Apple Silicon users should probably also open an issue for native binary builds on https://github.com/jgm/pandoc. That's a lengthy compile and uses a bunch of RAM and is mostly single-threaded.

[elbamos]

For what platform? And how does your Docker build differ from what ships with the RStudio source tree?

It runs R and RStudio Server with native ARM code rather than rosetta2 emulation. The performance difference is noticeable and significant.

The Apple Silicon users should probably also open an issue for native binary builds on https://github.com/jgm/pandoc. That's a lengthy compile and uses a bunch of RAM and is mostly single-threaded.

I linked you above to the discussion taking place among the homebrew and ghc folks regarding this. The issue has to do with the ghc compiler not supporting apple silicon yet. Once ghc supports apple silicon, binary distributions of pandoc will become available quickly.