Cannot install on MacbookAir M1 processor

[amarmureanuarnia]

➜  PROJECTS brew install davidalger/warden/warden
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/core and homebrew/cask).
==> Updated Formulae
Updated 40 formulae.
==> Updated Casks
Updated 9 casks.
==> Tapping davidalger/warden
Cloning into '/usr/local/Homebrew/Library/Taps/davidalger/homebrew-warden'...
remote: Enumerating objects: 210, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 210 (delta 13), reused 55 (delta 13), pack-reused 154
Receiving objects: 100% (210/210), 23.69 KiB | 1.08 MiB/s, done.
Resolving deltas: 100% (50/50), done.
Tapped 1 formula (14 files, 36.2KB).
Error: Cannot install in Homebrew on ARM processor in Intel default prefix (/usr/local)!
Please create a new installation in /opt/homebrew using one of the
"Alternative Installs" from:
  https://docs.brew.sh/Installation
You can migrate your previously installed formula list with:
  brew bundle dump

=====================
Running: MacOs Big Sur 11.2.3
Chip Apple M1

[davidalger]

@amarmureanuarnia Unless I'm mistaken, the error there is related to how Homebrew is installed and not specific to Warden.

Please create a new installation in /opt/homebrew using one of the
"Alternative Installs" from:
  https://docs.brew.sh/Installation
You can migrate your previously installed formula list with:
  brew bundle dump

On that linked installation docs page, you'll notice Homebrew states on M1 Macs it will install into /opt/homebrew instead of /urs/local and the error above indicates it's incorrectly installed, why I can't say, but I would look into that and try again.

If you want to give Warden a shot without needing to fix or change your Homebrew installation, feel free to use the alternate installation method documented here: https://docs.warden.dev/installing.html#alternative-installation

[amarmureanuarnia]

Thanks for the fast response. I already tried brew bundle dump but still not working. Probably I'm doing something wrong. I'll try the alternative way!

[drpayyne]

Hi @amarmureanuarnia, did you get a chance to get this working on an M1 Mac?

Also, @davidalger - is Warden officially supported on an M1 Mac? I'm just curious about the state of Magento development on Apple Silicon systems. :)

[amarmureanuarnia]

Hei @drpayyne ! Unfortunately, I didn't had time to try again. As of M1 Mac development, I can tell you that 90% of my work is still done on my old Macbook Pro late 2015 Intel based. I tried a couple of other projects with docker images, and from 4 projects only one worked without the need of doing some customization. Everyone that right now has an M1 Mac should invest time in building images for their project on the new ARM architecture. How I'm seeing it, is that even with the new Macbookpro 16 M2 there will still be missing pieces in the development process. In conclusion I still feel like an pioneer :)))

[davidalger]

There is certainly room for some improvements on M1 currently, but complete support (i.e. no emulation required) is going to be a long time coming and there is no ETA for that, as many of these issues relate to upstream deps, many of which lack arm64 / aarch64 images and/or rpms. I'm also limited in the time I have to spend building, testing, etc.

• There are no arm64 images for MySQL, Mailhog, or the base image used for Magepack image builds
• There are arm64 images for MariaDB, Nginx, RabbitMQ and Redis, but the build process in here needs amending to pull/build/push with both architectures
• Varnish doesn't even provide RPMs for arm64 so building a varnish image with native M1 support is not currently possible
• Elasticsearch publishes arm64 images (at least for later versions) but they use tags to denote this instead of multi-platform images as Docker Hub does, so ES specific build process changes will be needed to pull/build/push with both architectures.
• Remi does not build aarch64 RPMs for PHP, and currently has no ETA (or plans really) to do so as he lacks an arm builder and rebuilding PHP for Arm is not exactly a small endeavor (https://forum.remirepo.net/viewtopic.php?pid=11836#p11836) so the PHP-FPM images used here won't build for arm64 / aarch64 architecture (I tried it, which is how I discovered this)

My honest take is this: Docker Desktop literally just reached "GA" (whatever that means) only this past April. It was in tech preview starting Dec 2020, and that is hardly time to become stable. Developers that know they rely on Docker Desktop racing out to buy M1 macs, many before Docker Desktop was even out of tech preview, will necessarily be accepting some level of pain in exchange for being early adopters. There is a reason the first M1 models Apple has released are not super developer focused, it takes time (a lot of time) for an entire ecosystem to catch up, particularly with complicated things like emulation and virtualization.

Even with native arm64 images (that won't require QEMU emulation), there are "known issues" such as the following turning up in the release notes for Docker Desktop (this one is from June 6th):

On Apple Silicon in native arm64 containers, older versions of libssl in debian:buster, ubuntu:20.04 and centos:8 will segfault when connected to some TLS servers, for example curl https://dl.yarnpkg.com. The bug is fixed in newer versions of libssl in debian:bullseye, ubuntu:21.04 and fedora:35.

There is list of known issues (that I assume is kept current) here: https://docs.docker.com/docker-for-mac/apple-silicon/

An initial and first step here will be updating the image build system here to build and push multi-platform images for those where upstream dependencies exist to support it, while keeping the other image builds unchanged. This will take time to accomplish (and is something for which I also have no ETA as I cannot make it a priority at the moment)

[drpayyne]

Thank you so much @davidalger, for taking the time and effort for this detailed explanation of the state of Docker based Magento development in M1 Macs. This is the most helpful. It really gives a perfectly clear status of this.

This will take time to accomplish (and is something for which I also have no ETA as I cannot make it a priority at the moment)

Of course, I understand it will take a lot of time for full support. Thank you again for your work with Warden!

[davidalger]

Varnish doesn't even provide RPMs for arm64 so building a varnish image with native M1 support is not currently possible

Looks like they recently published aarch64 RPMs for 6.0lts, 6.5 and 6.6:

https://packagecloud.io/varnishcache/varnish60lts?filter=rpms
https://packagecloud.io/varnishcache/varnish65?filter=rpms
https://packagecloud.io/varnishcache/varnish66?filter=rpms

[drpayyne]

Elasticsearch publishes arm64 images (at least for later versions) but they use tags to denote this instead of multi-platform images as Docker Hub does, so ES specific build process changes will be needed to pull/build/push with both architectures.

Elasticsearch has multi-arch images in Docker Hub for versions >=7.8.0. Is this what we need? (ref: https://hub.docker.com/_/elasticsearch?tab=tags&page=1&ordering=name)

There are no arm64 images for MySQL, Mailhog, or the base image used for Magepack image builds

• MariaDB is Magento supported, so can we switch to that? Also, we don't have any customization in any of our MariaDB Dockerfiles, so is there a reason we have custom builds for that? If it's better to use the base MariaDB image directly, can we?
• MailHog hasn't been updated in over a year and here are our options
  • we contact its maintainer and check if it's on their roadmap (helps with our timelines)
  • if it's on their roadmap, we wait for them to push an arm build
  • if not
    • we use their Dockerfile and build it on our own for multi-arch considering its dependencies and publish it under the wardenenv Docker Hub namespace
    • we switch to https://hub.docker.com/r/sj26/mailcatcher which is both multi-arch and maintained. Relevant PR in @markshust's repo by me can be found here: [ARM Support] Replaced mailhog/mailhog with sj26/mailcatcher markshust/docker-magento#512. This is a proposal to switch MailHog with MailCatcher.
• There is an open issue (Crashes on Apple Silicon Zenika/alpine-chrome#152) for Magepack's base image and I've checked in to see if a multi-arch image is in their plans.

@davidalger, the above are my thoughts on this and I'm happy to collaborate and contribute to this as required. Please let me know what you think, thanks!

[fooman]

Might also be an option to switch the Mailhog image to
https://hub.docker.com/r/cd2team/mailhog/tags?page=1&ordering=last_updated

[drpayyne]

Update

I have forked this repository and made it run natively on both Intel and Apple Silicon chips, making it multi-platform. You can find more info at https://github.com/drpayyne/warden-multi-arch. Currently, only the one mainstream version of each service is rebuilt and provided.

I have rebuilt all necessary Dockerfiles as multi-platform Docker images (linux/arm64, linux/amd64) and published them in GitHub Packages.

Warden services

• I have re-used the MailHog Dockerfile and rebuilt it as a multi-platform image. Same for mhmailsend, which MailHog needs. Refer https://github.com/drpayyne/mhsendmail.
• Dnsmasq and sshd are rebuilt as multi-platform images. Refer https://github.com/drpayyne/docker-sshd and https://github.com/drpayyne/docker-dnsmasq-multi-arch.

Magento 2 services

• All PHP images have been switched to Fedora 34 from CentOS 8 after much efforts, as I tried to rebuild the entire PHP 7.4 binary and its extensions from Remi's repository for arm64 but was not successful as I don't have much experience in RPM building. Sorry! Refer https://github.com/drpayyne/docker-php.
• The base images for Magepack (Chrome with Node & Puppeteer) is rebuilt as multi-platform images. Refer https://github.com/drpayyne/chrome.
• Varnish 6.5 is rebuilt as a multi-platform image as Varnish now provides RPMs for both architectures.
• Elasticsearch 7.9 is rebuilt as a multi-platform image.
• Nginx 1.16 is rebuilt as a multi-platform image.
• RabbitMQ 3.8 is used directly from DockerHub. (already provides multi-platform images)
• All database containers now use MariaDB 10.4 directly from DockerHub. (already provides multi-platform images)
• Redis 5.0 is used directly from DockerHub. (already provides multi-platform images)

cc/ @fooman @davidalger @amarmureanuarnia

[erikhansen]

@drpayyne That's exciting! I'm looking forward to getting an M1 Macbook Pro once they're released (hopefully later this year) and will look into your fork at that time.

[DavidLambauer]

The fork from @drpayyne works fine for except xdebug, but there is already a pull request. Would be nice if you'd can consider to add the fork as an m1 version for warden or so

[drpayyne]

@DavidLambauer Just merged and rebuilt the Docker images. Thank you!

[stale]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[DavidLambauer]

Still valid, yes.

[dusandikovic]

I installed warden from warden-multi-arch fork on my Mac Book M1 Pro and when I do warden shell it logs me in, but I am not able to run any command from the terminal

It looks it doesn't recognize enter and doesn't run the command.

Also when I copy some text (example some copied text) from the host to container's terminal it adds brackets before and after copied text.

Did anyone have this problem?

[sprankhub]

Maybe you want to have a look at #501.

[Vitaliy-Lazarev]

All the magento commands in Warden are very, very slow on my M1 Pro and also the online store is slow. Do you know what could be the reason?

[DavidLambauer]

This project is not maintained anymore. Feel free to migrate to den --> https://github.com/swiftotter/den

den is a fork from warden, providing a faster, maintained version of warden!

[janosmiko]

You can also give https://github.com/rewardenv/reward a try. It's Warden rewritten in Golang.

Find the complete documentation here: https://rewardenv.readthedocs.io

[navarr]

New Apple silicon images in Beta: https://github.com/orgs/wardenenv/discussions/616

[pawel-snowdog]

This project is not maintained anymore. Feel free to migrate to den --> https://github.com/swiftotter/den

den is a fork from warden, providing a faster, maintained version of warden!

Hi. Is it safe to migrate from Warden https://github.com/drpayyne/warden-multi-arch to Den using standard method described in documentation? https://swiftotter.github.io/den/changelog.html

[bap14]

@pawel-snowdog You can use Warden 0.13 or greater and run it on an M1 mac. Den has since been deprecated in favor of Warden. Not all Den features have been ported over, but it's being worked on.

[pawel-snowdog]

@bap14 thx for info. Is there php8.2 and OpenSearch support in Warden newest release?

[navarr]

Warden package versions are available to browse both on Docker Hub and on GitHub: https://github.com/orgs/wardenenv/packages?repo_name=images

Latest PHP: 8.2
Latest OpenSearch: 2.6

I check for newer versions of all needed softwares semi-often

[erikhansen]

@pawel-snowdog I highly recommended moving to https://github.com/wardenenv/warden/, as @navarr is doing a great job maintaining it. Our team has moved from both Den and the https://github.com/drpayyne/warden-multi-arch project to Warden without any issue.

[pawel-snowdog]

@erikhansen great! Is there documentation how to migrate from https://github.com/drpayyne/warden-multi-arch to https://github.com/wardenenv/warden/ ?

[drpayyne]

Quick update - I've now archived the https://github.com/drpayyne/warden-multi-arch repository with a deprecation notice in favor of migrating back to this repository to avoid confusion.

[erikhansen]

@drpayyne Awesome.

Thanks for your early work in getting Apple Silicon compatibility—that enabled me to buy an M1X MacBook Pro a couple of years ago and enjoy the 2-4x M2 local development performance it offers over my Intel MacBook Pro.

[navarr]

Huge Appreciation for your work @drpayyne - I couldn't have lived without it ❤️

[pawel-snowdog]

Guys I need Your help with migration from https://github.com/drpayyne/warden-multi-arch to https://github.com/wardenenv/warden/ ? How to do it without loosing existing locals? Many thanks for your replies.

[drpayyne]

Thanks folks! Appreciate it. :) And thanks to you both back for the early testing and the continued support for Warden!

[erikhansen]

@pawel-snowdog

How to do it without loosing existing locals? Many thanks for your replies.

I did not have to do anything special when moving from https://github.com/drpayyne/warden-multi-arch to Warden, but YMMV. Summary of my steps:

warden svc down
# Shut down Docker Desktop (I'm using 4.10.0)

# Move warden to temp directory and then set reminder to delete this directory a few weeks later, once warden is still working
sudo mv /opt/warden /opt/warden_backup

brew install wardenenv/warden/warden

# Open Docker Desktop

warden svc up
warden env up