No ARM images available for development on Macbook Pro M1 CPU

[asim-blueprintprep]

Preconditions

1. Macbook Pro M1 cpu
2. php 7.3
3. redis 5
4. elasticsearch 7.5 (attempted)
5. varnish (attempted)

Steps to reproduce

1. ./vendor/bin/ece-docker build:compose
2. docker-compose up -d
3. multiple issues arise

Expected result

1. docker containers should be created, and all start with no errors

Actual result

1. elasticsearch issue
   1. Error: could not find libjava.so Error: Could not find Java SE Runtime Environment
   2. No ARM elasticsearch image provided by magento here: https://registry.hub.docker.com/r/magento/magento-cloud-docker-elasticsearch
   3. workaround, to use elasticsearch own here, https://registry.hub.docker.com/_/elasticsearch?tab=tags&page=1&ordering=last_updated
      1. e.g image: 'elasticsearch:7.8.1'
2. varnish issue
   1. varnish_1 | Error: varnish_1 | Running VCC-compiler failed, signal 5 varnish_1 | VCL compilation failed
   2. There are no ARM varnish images available in magento or varnishes own docker repos

Is there a known way of getting cloud docker to boot on a new Macbook Pro?

Or is there a way to disable varnish, and have the docker just go straight to the frontend node to serve the site without a FPC?

[shiftedreality]

@asim-blueprintprep thank you for this request.

We've created an internal issue https://jira.corp.magento.com/browse/MCLOUD-7989 to get it fixed.
However, any help is appreciated!

[asim-blueprintprep]

@shiftedreality thanks for repsonding so swiftly, do you know of a way of disabling varnish through docker compose, and configuring it so the site loads without varnish? As this is currently blocking development work

[drpayyne]

Hi @shiftedreality, is there any ticket to track the dependencies of cloud-docker and their compatibility with the Apple M1 chip? Just so that it's visible on what's pending for us Magento developers to start using cloud-docker on our Apple M1 computers.

[shiftedreality]

@asim-blueprintprep please use --no-varnish option

[shiftedreality]

@drpayyne I'm aware only about this one. Is there anything else we missing?

[drpayyne]

Not that I know of, @shiftedreality. Thanks for the update. Let me ask the community to see if anyone caught anything else.

[asim-blueprintprep]

Hi @shiftedreality
That created great, and allowed me to get docker-compose up -d booting all containers, I then run bash ./mutagen.sh
And then try to run docker-compose run --rm deploy cloud-deploy

However, it errors: [2021-06-16 14:48:55] ERROR: [132] Can't get version of elasticsearch: cURL error 6: Could not resolve host: elasticsearch (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

I can't help but wonder, if this is because I've changed the elasticsearch image to not be a magento-elasticsearch image?

[asim-blueprintprep]

I believe the issue above is resolved by having elasticsearch in your hosts file, after adding that and rebuilding that works. However, because docker desktop is having to emulate so many amd64 packages, it runs out of memory

docker/for-mac#5204

When running docker-compose run --rm deploy cloud-deploy

It reaches:
[2021-06-17 08:49:41] INFO: Running setup upgrade.
[2021-06-17 08:49:58] ERROR: [126] The command "/bin/bash -c "set -o pipefail; php ./bin/magento setup:upgrade --keep-generated --ansi --no-interaction | tee -a /app/var/log/install_upgrade.log"" failed. mmap() failed: [12] Cannot allocate memory

This is with 6GB RAM set available for docker desktop, I tried increasing it to 12GB, and the same error happened (with qemu actually using 16GB RAM - taken from top command)

So far:

1. varnish & elastic are the 2 containers that don't compile properly, even under emulation of amd64
2. docker runs out of memory trying to emulate all of the containers as ARM instead of amd64

I am out of ideas on how to progress this any further, without there being any ARM builds available

Thanks,

[drpayyne]

Hi @shiftedreality! Is there any update available for public view on this? :)

[WingmanImd]

Hey @drpayyne I have a working docker-compose.yml file in case you still need it. You'll need to keep the Dockerfiles in the project(I don't plan on pushing them to docker hub) though

[drpayyne]

Awesome @WingmanImd! Do you have this in your fork?

[WingmanImd]

@drpayyne no, I had to resort to manually editing the docker-compose.yml file after it's created then use modified versions of Dockerfiles for all images. Gonna fork it again and edit the Readme with the instructions later today

[antoinette-mkwapatira-at-hanes]

@WingmanImd Can you share a little more info about your approach to getting set up with the M1 chip?

[AlexanderKopyl]

Hello guys any one resolved this issue?

[BaDos]

Hi!
We're going to continue work on this issue soon.

[self-diagnostics]

Is there any news on this topic?

[MaximGns]

@shiftedreality any progress here?

[mustdobetter]

@BaDos I head up C3, an Adobe Commerce partner agency in the UK, and this is something I and my team need. I'm keen to know if I can be of help to move this forward - we have lots of Docker experience.

[mustdobetter]

I've just found a commit adding support: 50c77b0. Thank you @BaDos! 🙌

I've successfully built and run the PHP 7.4 FPM image on ARM (MacBook Pro Max), and all seems to be working well. I'll be building and testing some of the others shortly and will feed back on any issues etc.

[ferroapp]

Hi, I wonder if there is any progress on this, we are stuck in the same situation, version 2.4..4, any help will be much appreciated

[BaDos]

Hi @ferroapp
Now we have prepared Dockerfiles to build images for arm64/M1, but we do not have images on docker hub.

[ferroapp]

@BaDos that's nice, what version should we try?? 1.3.3?? thank you

[BaDos]

@ferroapp yes. you can use 1.3.3

[mamarx]

really happy about the ARM64 support added in 1.3.3. thank you guys!

we updated to 1.3.3 and built a new docker setup but still AMD64 images. what do we have to update to use the ARM64? thanks in advance!

[BaDos]

@mamarx docker hub contains only amd64 images, you need to build you own arm64 images using M1 machine or any arm64

[mamarx]

@BaDos thank you!

[ga3003]

Is there any update on this issue? I am using the Mac Pro M2 and I am facing the same issue where I got a warning for ARM images and when the setup is complete, the URL is showing an error message of not reachable.

Please help, if someone has any hack to solve this.

[BaDos]

Hi @ga3003
Please, see comments above.
Now you need to build arm64 images yourself.

[ga3003]

Thanks, @BaDos.

[pawelskowronek]

Here is an example of how you can build arm64 images for Magento Cloud Docker 1.3.4 and Magento 2.4.5-p1:

1. Clone magento cloud docker repository.

   git clone https://github.com/magento/magento-cloud-docker.git

2. Change current working directory to magento cloud docker.

   cd magento-cloud-docker

3. Build docker images.

   docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-php:8.1-cli-1.3.4 ./images/php/8.1-cli
   
   docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-opensearch:1.2-1.3.4 ./images/opensearch/1.2
   
   docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-nginx:1.19-1.3.4 ./images/nginx/1.19
   
   docker image build --platform=linux/arm64/v8 -t magento/magento-cloud-docker-varnish:6.6-1.3.4 ./images/varnish/6.6
   

[IgorVitol]

@mamarx docker hub contains only amd64 images, you need to build you own arm64 images using M1 machine or any arm64

@BaDos Hi! Is there any plans to build the cross-platform images and push those directly to Docker Hub to prevent any manual "build" actions by every dev? Same name/tag would be used on all platforms letting docker to download the proper platform image automatically.

This should be as simple as configuring your build pipeline to use proper BuildKit in dockerx, eg
docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 -t <username>/<image>:latest --push

Note: You can see example in varnish/redis official images, which has inbuilt cross-platform support:

[joeshelton-wagento]

Any updates here? I'm able to locally build arm64 images. But, I can't utilize them without a manual modification of the docker-compose.yml file. (And changing permissions on the executable files copied to the image.) These manual modifications are not easy to figure out, and don't persist when updating the tool. So, not ideal for new Mac users.

[JonasWijne]

writing on jan 22 2024, is there any update?

[drpayyne]

I wonder what's stopping the Adobe team from developing and publishing this? How are the 100s of internal Adobe developers working on Magento on their MacBook? Either everyone has the old chips, or they're all running these containers on emulation, or their multi-platform images are proprietary. :)

[Morgy93]

What a great developer experience in 2024 with latest technology. Makes me want to jump on Magento Cloud right away!

Or that's what I'd like to say if this issue would not exist. Any news?