Apple Silicon and basic cross compiling support #26

brainstorm · 2021-08-24T04:39:25Z

Followup from aws/aws-sam-cli#3132, refer to that issue and @jfuss for context.

This PR adds the possibility to build the docker images on Apple Silicon, by activating the environment variable:

$ export SAM_CLI_CROSS_BUILD=1

Otherwise none of the containers in build-image-src/build_all_images.sh will successfully build on an Apple M1 machine (Darwin/arm64). Some of the errors of not including buildx in the docker image building process include the following, predictably:

(...)
#8 43.57 ---> Package binutils.aarch64 0:2.29.1-30.amzn2 will be installed
#8 43.57 --> Processing Dependency: libdl.so.2(GLIBC_2.17)(64bit) for package: binutils-2.29.1-30.amzn2.aarch64
#8 43.57 Package: glibc-2.26-48.amzn2.aarch64 - can't co-install with glibc-2.26-48.amzn2.x86_64
#8 43.58 --> Processing Dependency: ld-linux-aarch64.so.1(GLIBC_2.17)(64bit) for package: binutils-2.29.1-30.amzn2.aarch64
#8 43.58 Package: glibc-2.26-48.amzn2.aarch64 - can't co-install with glibc-2.26-48.amzn2.x86_64
#8 43.58 --> Processing Dependency: ld-linux-aarch64.so.1()(64bit) for package: binutils-2.29.1-30.amzn2.aarch64
#8 43.58 Package: glibc-2.26-48.amzn2.aarch64 - can't co-install with glibc-2.26-48.amzn2.x86_64
(...)

…ded.al2 image

brainstorm · 2021-09-28T03:17:41Z

@hoffa Could you review/test this please?

hoffa · 2021-10-07T22:08:53Z

Hi @brainstorm! Thanks for this PR. We've published ARM images for a number of runtimes (in line with what's supported for Lambda). Does that help with your use case? Recent ARM-related changes to build_all_images.sh might make supporting multi-platform builds generically slightly less straightforward.

brainstorm · 2021-10-08T03:35:38Z

It does! 🎉

Thanks a ton @hoffa for this invaluable pointer! Now umccr/s3-rust-noodles-bam@81a8f0f...6bb853e solves a longstanding issue for me, much appreciated! Now I can deploy those lambdas on ARM64 and ship aws-sdk-rust with its tricky ring dependency with SAM-CLI /cc @nmoutschen

Unfortunately the sam local start-api story remains elusive, see https://github.com/umccr/s3-rust-noodles-bam/blob/s3-server/FLUKES.md for how it fails on my Apple M1 :/

brainstorm · 2021-10-08T03:55:45Z

The exec format error is specially confusing because I'm building everything as arm64/aarch64 now, no x86_64 or similar:

(base) rvalls@m1 s3-rust-noodles-bam % find . -iname bootstrap | xargs file
./target/debug/bootstrap:                                              Mach-O 64-bit executable arm64
./target/debug/deps/bootstrap:                                         Mach-O 64-bit executable arm64
./target/debug/deps/bootstrap.dSYM/Contents/Resources/DWARF/bootstrap: Mach-O 64-bit dSYM companion file arm64
./.aws-sam/build/s3Bam/bootstrap:                                      ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=cf3b3e130123119762ee6b373d6e93c6326f3667, with debug_info, not stripped

What am I doing wrong? :-S

brainstorm · 2021-10-08T05:51:43Z

Also, as a side note, the local building performance is quite disappointing since I guess it doesn't leverage the local machine's .cargo cache (6 minutes of build on a powerful M1, takes seconds locally without this docker-based sam-cli build :-!!):

% time sam build -u -bi provided.al2-rust
Starting Build inside a container
Building codeuri: /Users/rvalls/dev/umccr/s3-rust-noodles-bam runtime: provided.al2 metadata: {'BuildMethod': 'makefile'} architecture: ARM64 functions: ['s3Bam']
Failed to download a new provided.al2-rust image. Invoking with the already downloaded image.
Mounting /Users/rvalls/dev/umccr/s3-rust-noodles-bam as /tmp/samcli/source:ro,delegated inside runtime container

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

Running CustomMakeBuilder:CopySource
Running CustomMakeBuilder:MakeBuild
Current Artifacts Directory : /tmp/samcli/artifacts
sam build -u -bi provided.al2-rust  0.45s user 0.19s system 0% cpu 6:02.99 total

This is a perfect example of one more paper cut when it comes to DX, see: aws-samples/serverless-rust-demo#4

brainstorm · 2021-10-08T09:34:39Z

@hoffa Here's another weird one... would you like me to open separate issues for these last three findings?:

% sam deploy
File with same data already exists at s3-rust-noodles-bam/35dc3dcb4269a08f321c2807a3a4e7fb, skipping upload

	Deploying with following values
	===============================
	Stack name                   : s3-rust-noodles-bam
	Region                       : ap-southeast-2
	Confirm changeset            : True
	Deployment s3 bucket         : aws-sam-cli-managed-default-samclisourcebucket-1v8v4fkgfc6w1
	Capabilities                 : ["CAPABILITY_IAM"]
	Parameter overrides          : {}
	Signing Profiles             : {}

Initiating deployment
=====================
Uploading to s3-rust-noodles-bam/05e1495bce7efea2d0e7ff36051fd8c1.template  880 / 880  (100.00%)

Waiting for changeset to be created..
Error: Failed to create changeset for the stack: s3-rust-noodles-bam, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state: For expression "Status" we matched expected path: "FAILED" Status: FAILED. Reason: Transform AWS::Serverless-2016-10-31 failed with: Invalid Serverless Application Specification document. Number of errors found: 1. Resource with id [s3Bam] is invalid. Architectures needs to be a list with one string, either `x86_64` or `arm64`.
(base) rvalls@m1 s3-rust-noodles-bam % find . -iname "*.yaml" | xargs grep "Architecture"
./template.yaml:      Architectures: ["ARM64"]
./.aws-sam/build/template.yaml:      Architectures:
(base) rvalls@m1 s3-rust-noodles-bam % find . -iname "*.yaml" | xargs grep -C3 "Architecture"
./template.yaml-      Handler: bootstrap.binary.is.real.handler
./template.yaml-      Runtime: provided.al2
./template.yaml-      MemorySize: 128
./template.yaml:      Architectures: ["ARM64"]
./template.yaml-      Timeout: 30
./template.yaml-      CodeUri: .
./template.yaml-      Policies:
--
--
./.aws-sam/build/template.yaml-      Handler: bootstrap.binary.is.real.handler
./.aws-sam/build/template.yaml-      Runtime: provided.al2
./.aws-sam/build/template.yaml-      MemorySize: 128
./.aws-sam/build/template.yaml:      Architectures:
./.aws-sam/build/template.yaml-      - ARM64
./.aws-sam/build/template.yaml-      Timeout: 30
./.aws-sam/build/template.yaml-      CodeUri: s3Bam

brainstorm · 2021-10-11T05:02:06Z

@hoffa, nevermind the last three messages, I managed to have it running on umccr/s3-rust-noodles-bam@5d2ce7a ;)

Even the sam local start-api works now: umccr/s3-rust-noodles-bam@4953487

Although I now I have figure out if I should define a dummy credential provider chain for S3, otherwise I'm getting:

Invalid lambda response received: Invalid API Gateway Response Keys: {'errorType', 'errorMessage'} in 
{'errorType': '&alloc::boxed::Box<dyn std::error::Error+core::marker::Send+core::marker::Sync>', 
'errorMessage': 'failed to construct request: No credentials in the property bag'}

I guess that there are two problems here:

sam local start-api is not mounting/seeing my host's ~/.aws/credentials, so it's not able to access the (non-public) S3 bucket I'm pointing to.
AWS SSO, which is the authentication my org uses, is not yet supported by aws-sdk-rust (currently using 0.0.19)? /cc @jdisanti?

From aws-sdk-rust README:

I guess that for now the only workaround possible to address local debugging with auth'd resources from my org is using the following with sam local start-api?:

  -n, --env-vars PATH             JSON file containing values for Lambda
                                  function's environment variables.
  --container-env-vars PATH       JSON file containing environment variables
                                  to be set within the container environment

brainstorm · 2021-10-12T05:04:37Z

@hoffa Credentials issue mentioned in the last message was trivial to solve via DefaultChainProvider (my bad, I didn't remember that the lambda has all required AWS env vars defined and ready to use already, so no SSO required).

What I don't know how to address properly (yet), and it's quite a DX nuisance, is the aforementioned build times on Apple Silicon of 6 minutes via my own rust-provided.al2 container:

$ docker build -t provided.al2-rust . -f Dockerfile-provided.al2
$ sam build -c -u --skip-pull-image -bi provided.al2-rust
$ sam deploy

The step in the middle takes too long for a normal developer iteration loop... @nmoutschen is trying to use the rust-embedded cross tool (which coincidentally I also used in the past), but IMHO that will not solve the long sam build times unless AWS SAM CLI implements caching for Rust, mounting .cargo/cache on the container to avoid re-compiling everything from scratch inside it.

Alternatively, if somebody finds a way to cross-compile ring on M1 natively (as in using the host's cargo build and no docker), that would be perfect DX.

IMHO, I very much prefer the latter option to have an acceptable local build time, but it doesn't look straightforward to solve ATM, at least, to me and my limited time to dedicate to AWS tooling :-S

brainstorm added 4 commits August 24, 2021 14:31

Apple Silicon and basic cross compiling support

19a41b4

Revert rust-specific changes that do not belong to the upstream provi…

4879e93

…ded.al2 image

Put ENV PATH back

e9a95c1

Merge branch 'develop' into apple_silicon_cross_builds

5fcdcf6

brainstorm closed this Oct 8, 2021

brainstorm deleted the apple_silicon_cross_builds branch October 8, 2021 03:43

brainstorm mentioned this pull request Oct 8, 2021

"sam build -u -e TARGET_CC=foo" leads to "cargo: Command not found" aws/aws-sam-cli#3132

Closed

brainstorm mentioned this pull request Oct 11, 2021

"include/GFp/check.h:27:11: fatal error: 'assert.h' file not found" on M1 apple silicon computers briansmith/ring#1332

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apple Silicon and basic cross compiling support #26

Apple Silicon and basic cross compiling support #26

brainstorm commented Aug 24, 2021 •

edited

Loading

brainstorm commented Sep 28, 2021

hoffa commented Oct 7, 2021

brainstorm commented Oct 8, 2021

brainstorm commented Oct 8, 2021

brainstorm commented Oct 8, 2021 •

edited

Loading

brainstorm commented Oct 8, 2021

brainstorm commented Oct 11, 2021 •

edited

Loading

brainstorm commented Oct 12, 2021 •

edited

Loading

Apple Silicon and basic cross compiling support #26

Apple Silicon and basic cross compiling support #26

Conversation

brainstorm commented Aug 24, 2021 • edited Loading

brainstorm commented Sep 28, 2021

hoffa commented Oct 7, 2021

brainstorm commented Oct 8, 2021

brainstorm commented Oct 8, 2021

brainstorm commented Oct 8, 2021 • edited Loading

brainstorm commented Oct 8, 2021

brainstorm commented Oct 11, 2021 • edited Loading

brainstorm commented Oct 12, 2021 • edited Loading

brainstorm commented Aug 24, 2021 •

edited

Loading

brainstorm commented Oct 8, 2021 •

edited

Loading

brainstorm commented Oct 11, 2021 •

edited

Loading

brainstorm commented Oct 12, 2021 •

edited

Loading