Platform Support x86_64 & aarch64

[metaskills]

Might have to change our Dockerfile around a bit if not Tier 1. More details here. https://doc.rust-lang.org/nightly/rustc/platform-support.html

[jhuiting]

@metaskills Any updates on the plans for arm64? Would add a lot of value to have both architectures supported. Really nice work on the extension, always was hoping that AWS made it this seamless to integrate secrets!

[metaskills]

No, I have yet to crack the nut on the Codespaces GPU Beta and/or GitHub Actions support for arm64 or how those might be needed or avoided. It is on my list tho. Do you have experience or thoughts here? Would really appreciate the help.

[jhuiting]

@metaskills I also didn't know but seems there's no native way to run ARM64 yet. But cross-compile seems to be quite simple based on cross (sample with the Cargo Github action: https://github.com/jhuiting/rust-cross-compile-test/blob/master/.github/workflows/build.yml).

[mattbajorek]

cargo-lambda has worked for me to build for arm64 on Github actions for any Rust project. It uses cargo-zigbuild under the hood and creates a folder with the project name and a binary inside named bootstrap. I have not tried their lambda extensions though, which may be a better use case for this project.

For regular builds I would do:

      - name: Install Cargo lambda
        run: pip3 install cargo-lambda

      - name: Cargo build
        run: cargo lambda build --release --arm64

[metaskills]

I recently did some work in the Lamby cookiecutter to build a multi-platform image so the Python cookiecutter library worked on arm, see here: https://github.com/customink/lamby-cookiecutter/blob/master/.github/workflows/cc-in-docker.yml

I was surprised how easy this was. So I think there is a mix of the cargo work above with this. I'd love to work or see a PR soon. Seems really straightforward.

[metaskills]

I've started working on this so I can ensure the latest Lamby docs around this are right. I want folks to easily ship arm64 with a few lines of change. Right now, I think there is a blocker with Crypteia not working on arm64.

• Outline Arm64 Steps lamby#143

[metaskills]

Few bookmarks:

• https://www.linkedin.com/pulse/testing-alternative-c-memory-allocators-pt-2-musl-mystery-gomes/
• https://github.com/microsoft/mimalloc
• https://crates.io/crates/cross
• https://github.com/multiarch/qemu-user-static
• https://www.stereolabs.com/docs/docker/building-arm-container-on-x86/
• https://www.docker.com/blog/faster-multi-platform-builds-dockerfile-cross-compilation-guide/

[metaskills]

Can folks try out the new v1.1.0 images? They should automatically point to the proper architecture. Meaning if you are building an arm64 image, you should get the right files, binaries, and shared objects.

Amazon Linux 2

docker manifest inspect ghcr.io/customink/crypteia-extension-amzn:1.1.0
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 528,
         "digest": "sha256:662e8f43106cd7685ca914d3ea9506fd34ee9b625e75583c869e7e83316decd8",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 528,
         "digest": "sha256:26fec765f44dc9292bca1fea18ac6acee33991d4406dc16f8b1f6726cf5a21da",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      }
   ]
}

Debian/Ubuntu

$ docker manifest inspect ghcr.io/customink/crypteia-extension-debian:1.1.0
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 528,
         "digest": "sha256:123c6b4edda68f215e229f70e248ddf4c07d891c1d701bed27d5f015ca66f9a5",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 528,
         "digest": "sha256:9befdde5fc24ecccb11bd38c8b526e205ebb3b4ea523cd92abb12ef0eb779471",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      }
   ]
}

[metaskills]

https://dev.to/aws-builders/using-docker-manifest-to-create-multi-arch-images-on-aws-graviton-processors-1320

[metaskills]

🤦‍♂️ I just noticed the manifest has amd64 twice. I'm not sure what is happening here and where the scripts/process went wrong.

$ docker inspect ghcr.io/customink/crypteia-extension-debian-arm64:1.1.0 | grep Arch
        "Architecture": "amd64",

[metaskills]

Will see if this helps. 78d56e8

[metaskills]

Interesting this got added to the devcontainer/ci project. Also, here is the link to base Linux ARM64 actions runner too.

• Add multi-platform builds devcontainers/ci#175
• Support Linux ARM64 build images actions/runner-images#5631

[metaskills]

So... v1.1.0 is done. Give it a whirl.