Support for Docker Desktop on Apple Silicon

[nebuk89]

Tell us about your request
Provide support for Docker Desktop for Apple computers running on Apple custom silicon.

Which service(s) is this request for?
Docker Desktop for Mac

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Mac will be moving to a new CPU architecture where Docker Desktop will no longer run, this ticket is to release Docker Desktop on the new architecture

Are you currently working around the issue?
Using a self hosted VMs on new Macs to run Docker

Additional context
https://www.apple.com/uk/newsroom/2020/06/apple-announces-mac-transition-to-apple-silicon/

[chris-crone]

Work on docker-compose can be tracked here: docker/compose#7945

[thaJeztah]

Progress on getting Docker Desktop running on Apple Silicon (M1). Lots (lots!) of duct-tape still, but this screenshot should make many people happy: (I can't take any credits for this, so give Dave's tweet a like ❤️ https://twitter.com/mugofsoup/status/1332382741892124675?s=20)

[rene-mueller]

@thaJeztah is there a download of that version to download and test?

[thaJeztah]

@rene-mueller no, much too early for that. This is a local build on a developer's machine. Still needs lots of hand-holding to run it, and we have a very limited number of machines to work with (remote-desktop across continents), waiting for more machines to arrive (also for our release and CI pipelines).

We definitely want to get test builds out as soon as possible, when they're more usable, but that will still take some time.

[rene-mueller]

@thaJeztah what a pity, is there any way to help you and speed up the process?

[thaJeztah]

Not at this moment. Team is working on the low-level stuff to get all components running. Debugging some crashes (which could be bugs in Golang (or the way it's used in our code), and bugs in macOS).

[EDemerzel]

@thaJeztah It certainly looks like you and the team are making good progress. Would it be possible for you to give us a realistic ETA on getting an alpha or beta that runs on the M1 in our hands? I will not hold you to it but it would help with planning. Thanks so much!

[infostreams]

@EDemerzel Not part of the team so I couldn't tell you, but I find your question/demand really super inappropriate. It's done when it's done. If you want to plan (for what?), assume early June and be happily surprised if it's finished early. Otherwise lay off on the pressure and be grateful that you will receive the fruits of their labor for free - whenever that might be. I don't think it's ok to add pressure by asking "when is it done?"

[KarimGeiger]

@infostreams Relax, okay? He asked politely and just wanted to know where things are at. He didn't pressure anyone, it was just a question to get a better understanding of what's currently going on. And, believe it or not, some people actually do need to do some planning, especially in an enterprise world. This isn't all fun and games, some people need to work, and some people need docker for this. And yes, docker is free, and he or anyone else using it for free can't demand anything to be done, but it's another thing to just ask a simple question.

[infostreams]

@KarimGeiger No. He is putting pressure and he is making demands, however politely phrased, and no - that's not ok. I will stop replying to this thread now, but this needed to be said. I don't mind being the scapegoat for that.

[EDemerzel]

@KarimGeiger No. He is putting pressure and he is making demands, however politely phrased, and no - that's not ok. I will stop replying to this thread now, but this needed to be said. I don't mind being the scapegoat for that.

Just for the record I wasn't attempting to place pressure or demands. I wasn't even asking about a RC or a Release. On most of my machines I run "edge" and enjoy testing and giving feedback when I can. I'm someone that is excited about the M1 and helping products move from Intel to Apple Silicon. Also, while I do not personally pay for Docker services the people I work for do so I'm not a freeloader. Lastly, I do apologize if my question rubbed you the wrong way.

[fieu]

Please guys, this is an issue related to a bug in the software. Please refrain from leaving a comment unless it contributes to the conversation at hand.

[andreiborisov]

Assuming we have Docker Desktop for Apple Silicon Macs, how much of a performance hit we’re talking about when running x86 images through Docker QEMU?

[thaJeztah]

It certainly looks like you and the team are making good progress. Would it be possible for you to give us a realistic ETA on getting an alpha or beta that runs on the M1 in our hands? I will not hold you to it but it would help with planning. Thanks so much!

At this moment, any estimation would be only "guessing" and very likely to be completely off. The honest answer is "we don't know, until we reached that point". To illustrate that; while we did have parts of Desktop running before, the screenshot was posted just minutes after an engineer was able to spin up Desktop (including the UI). There's still many parts that are currently running with duct-tape solutions (e.g. networking with hard-coded IP addresses etc). These builds are also built with unreleased versions of (build-time) dependencies, such as Golang (and I think with some temporary patches on top of that); while that works to get a test-build, that's not something we can use for an actual (regular) release.

This is pretty much "uncharted territory", and given that Docker tends to use parts of operating systems (including macOS) and kernels in very specific ways that other software may not be using, we may be (and often do) running into bugs in macOS that would need to be fixed before we're able to do a somewhat stable release.

I find your question/demand really super inappropriate
..
He asked politely and just wanted to know where things are at.

@infostreams @KarimGeiger thank you for stepping in; no worries: we're good at "reading between the lines", and I assume @EDemerzel's question was all in good faith.

We are all super-excited to see things progress (I for sure am eyeing the new M1 machines to give them a spin), and with engineers in both San Francisco and Europe working on this, we're literally working around the clock to get things working, and try to make that a reality.

And, believe it or not, some people actually do need to do some planning, especially in an enterprise world.

(Personal opinion); while it's definitely good to start testing with these machines in (enterprise) environments, I would not make a major switch to an entirely new hardware for business-critical situations on "day 1". These machines are the first generation of the Apple Silicon platform; while they should be good for may use-cases, there might still be things that need to be addressed. I very much expect that Apple started with the "lower range" models for this reason: allowing them (and users) to "test the waters" before proceeding with the higher-spec models.

Assuming we have Docker Desktop for Apple Silicon Macs, how much of a performance hit we’re talking about when running x86 images through Docker QEMU?

I'm not sure if we have good benchmarks yet. That said; these machines are FAST, and even with some overhead, performance may be comparable with native intel. Again; we don't have benchmarks yet (but we'll keep you posted once we have).

With all of the above said, I should add some big warnings ⚠️ to set expectations right;

• While we do want to get things into your hands as soon as realistically possible, releasing it "to the world" also brings "support" and "bug-reports" with it. With millions of Docker Desktop users, we wouldn't be able to deal with the "firehose" or reports. For that reason, it's possible we will send out test-builds to a limited group of users before making them generally available. (Nothing decided yet on that, and this will depend on how things progress, etc, etc). Also read the related blog-post which has a sign-up link for a Docker ID (and the newsletter).
• As mentioned; currently, we depend on either unreleased, or patched versions of (build-time) dependencies. Those are not suitable for regular releases, so while we may be releasing test-builds at some point (see above), "regular" releases may still be some time out.
• Since we started the work on Apple Silicon, we uncovered some bugs in macOS and the macOS kernel on these machines. Some may have been patched since, but others may either have to be worked around, or we have to wait for those to be fixed by Apple.
• Additional changes may still be needed in other components, which may include changes to "how" architectures are selected from multi-arch images for these systems.

[EDemerzel]

@thaJeztah This is wonderful information and certainly more than I had ever hoped! I really appreciate you taking the time to interact with the community like you do.

[Geekimo]

Thanks for giving us these informations. I'm part of thoses which received a M1 while having ordered an Intel one in early November, prior M1 unveiling. I hope all bugs will soon be fixed by cupertino.
Does anyone know if we can do something to run Docker on M1 inside some kind of VM ?

[FernandoMiguel]

Thanks for giving us these informations. I'm part of thoses which received a M1 while having ordered an Intel one in early November, prior M1 unveiling.

You ordered a new Intel based Mac in November and instead received an ARM one???

[finestructure]

Does anyone know if we can do something to run Docker on M1 inside some kind of VM ?

Yes: https://finestructure.co/blog/2020/11/27/running-docker-on-apple-silicon-m1

[Geekimo]

Thanks for giving us these informations. I'm part of thoses which received a M1 while having ordered an Intel one in early November, prior M1 unveiling.

You ordered a new Intel based Mac in November and instead received an ARM one???

Yes, we (the company I work for) ordered a MBP 13" in November 4th, and the day after M1 reveal the ordered model was switched from Intel to M1.

@finestructure Thanks, I was reading it when your message came up, nice work. 😉 I'll get a VM from Scaleway to host my developments during the meantime of the M1-compatibility development.

[aiur100]

Love you all for working so hard on this. Thank you!

[Kyngo]

Amazing work being done here! 👏🏻 👏🏻 👏🏻

[musikov]

Yet another comment to notify every subscriber. Thank you!

[smt116]

It might be a good idea to lock this discussion and allow only comments from the team. Otherwise, the topic's subscription does not make any sense.

/cc @nebuk89

[Tenzer]

Hey Apple M1 users,
we have a new preview ready for you.

Thanks for the update! As the preview builds don't auto-update, can we rely on comments here when new releases are available?

• The updated version includes a change that should improve disk performance.

The biggest problem I've had with preview 7 was the disk performance was poor, especially for MariaDB, which caused tests for my codebase to take three times as long as with a non-ARM64 Mac. With preview 7, I resorted to storing the database files on a tmpfs volume which helped massively. I hoped this update would help with this, but I do not see any noticeable difference with version 3.1.0 (60984).

To quantify this better, I tried to run some benchmarking with fio using the configuration from this Percona blog post since that seemed relevant for MySQL/MariaDB. The tests were run with size=100m to keep the time necessary to do this down. All tests were run inside an ubuntu:latest container with fio installed via apt-get.

Test scenario	ibd_sync_read	ibd_async_write	ib_log_sync_write
No volume	90 IOPS / 1.41MiB/s	90 IOPS / 1.41MiB/s	178 IOPS / 1.39MiB/s
Volume	87 IOPS / 1.41MiB/s	84 IOPS / 1.33MiB/s	167 IOPS / 1.31MiB/s
Bind mount	1188 IOPS / 18.6MiB/s	1778 IOPS / 27.8MiB/s	1947 IOPS / 15.2MiB/s

The bind mount testing was done using gRPC FUSE. I tried with osxfs, which showed roughly double the throughput for the ib_log_sync_write test, while the other tests failed with osxfs.

I also tried to use the :delegated option, but it didn't make any noticeable difference.

Running the tests directly in macOS and on a tmpfs volume were both much, much quicker but I left them out since it's not too relevant here.

I was surprised to see the bind mount's performance being so much better than the other options, as I expected that to be slowest.

Do you have any indication for where the bottleneck would be for disk IO? It would be interesting to try and run similar tests inside the VM that Docker daemon is running on to see if the problem is with Hypervisor.framework or higher up the stack.

[underfisk]

@Tenzer I do agree with you, in fact, its much faster but i think that was also expected but since Hypervisor framework got some adjustments for ARM now it will probably slow docker team due to the fact they have to support also this new arch

[stephen-turner]

We have seen some situations in which disk performance is worse on the virtualization framework than on the old hypervisor framework, and we are talking to Apple about it.

[underfisk]

@StefanScherer That is perfectly normal to happen, they are probably still adapting the kit to most of the apps that were relying on intel based. Is there anything new from Apple regarding hypervisor or it's still under prep talk?

[stephen-turner]

Nothing concrete yet.

[eveevans]

@Tenzer I had the same issue. My container response felt so slow on my M1.
I'll try with tmpfs. Also I'll try the new preview And I'll comment the results here.

[jan-runkel-zeitgleich]

Docker on M1 can run any linux/arm64 image out of the box, and any Intel image with emulation if you add --platform linux/amd64 to the command line.
EDIT: if someone can tell me how to specify that flag from a docker-compose.yml file, I'd be very grateful.

putting platform: linux/x86_64 in the docker-compose.yml did the trick for me.

can you try to start bitnami/apache on apple silicon?

# dont work
version: "3.8"

services:
  apache:
    image: bitnami/apache
    platform: linux/x86_64

I become this error on startup.

apache 09:58:33.42 INFO  ==> ** Starting Apache **
[Wed Feb 24 09:58:33.504807 2021] [core:emerg] [pid 1] (95)Operation not supported: AH00023: Couldn't create the mpm-accept mutex 
(95)Operation not supported: could not create accept mutex
AH00015: Unable to open logs

[Geekimo]

Hi @jan-runkel-zeitgleich ,
thanks for your input, but the platform flag is not in the 3.8 docker compose specification.
This is why we have to pass it through the command line.

[jan-runkel-zeitgleich]

Hi @jan-runkel-zeitgleich ,
thanks for your input, but the platform flag is not in the 3.8 docker compose specification.
This is why we have to pass it through the command line.

Is this the correct way to start a x86 image on Apple Silicon?

# dont work
docker run --platform linux/x86_64 bitnami/apache:latest

or this?

# dont work
docker run --platform linux/amd64 bitnami/apache:latest

Or is there another Way to start this image?
This two command dont work on my machine.

[DavidGarciaCat]

try with linux/arm64 as it's the way it works for me

EDIT:

I mean this:

docker run --platform linux/arm64 bitnami/apache:latest

[jan-runkel-zeitgleich]

try with linux/arm64 as it's the way it works for me

EDIT:

I mean this:

docker run --platform linux/arm64 bitnami/apache:latest

thanks for the fast response. I think I forgot something
with your docker run command following message pop up:

docker: Error response from daemon: image with reference bitnami/apache:latest was found but does not match the specified platform: wanted linux/arm64, actual: linux/amd64.

I install today the Docker Apple Silicon preview 7
I also install rosetta2.

softwareupdate --install-rosetta

Is there another Step required to use the x86 emulation?

[thaJeztah]

@jan-runkel-zeitgleich @DavidGarciaCat please don't use this ticket to report bugs or support questions; use https://github.com/docker/for-mac/issues for that (or the "dev-preview-program" slack channel if you're on that list)

[jan-runkel-zeitgleich]

You're right.
Unfortunately I don't have access to the Slack channel yet.
This ticket describes that with the flag "platform: linux / x86_64" Intel images should run. But that doesn't seem to work.

[thaJeztah]

This ticket describes that with the flag "platform: linux / x86_64" Intel images should run. But that doesn't seem to work.

The --platform flag works if the image is a multi-arch image (which is the case for all official images, but may not be the case for other images). The bitnami/apache image is not a multi-arch image, and only supports linux/amd64 (x86); https://hub.docker.com/r/bitnami/apache/tags?page=1&ordering=last_updated

When you specify --platform, docker will only pull the image if it finds a match (which, in this case, it doesn't).

As a workaround you can try pulling it without --platform, which (in case of a non-multi-arch image) will make docker "ignore" the platform, and just pull "whatever it found".
But, it would still be an x86 image that you run on an arm platform, which means it will try to run it using qemu emulation, which should be considered a "best effort" and may work (but no guarantee, as there's still many bugs/known issues in qemu).

We will continue looking to improve the emulation, and are also in talk with 🍏, but it will remain a "best effort", and mostly to help the transition to ARM becoming more mainstream (both on hosting platforms, and in images).

Because of the above, the recommendation is to run images that match the architecture.

I see you opened https://github.com/bitnami/bitnami-docker-apache/issues/104, which is probably best for that

[DavidGarciaCat]

Hello,

Just aiming to report a potential bug:

When I download and install the Docker Preview on my M1, it seems to run. If I open the Docker Desktop Preferences window and set Docker not to start when booting up my laptop, then I am unable to use it again, and I need to remove it and install it again.

The error message I get is that my M1 is not compatible to use Intel software, even when I already used it and Rosetta is installed.

Maybe this is something to be reviewed and addressed in an upcoming update?

Cheers,

[jonareyes]

hi everybody, anybody has a solution to inotify issue, I have RC3 Docker Desktop but I don't have some fix o a temporary fix, can you help me, anybody?

[lemire]

When I download and install the Docker Preview on my M1, it seems to run. If I open the Docker Desktop Preferences window and set Docker not to start when booting up my laptop, then I am unable to use it again, and I need to remove it and install it again.

It does seem that if I shut down Docker Desktop, it won't start again unless I reboot.

[finestructure]

I frequently see the same issue, especially after wake from sleep. Going into Activity Monitor and quitting the Docker process(es) lets me restart. Should save you the reboot.

[nebuk89]

As this is now shipped 🎉 https://www.docker.com/blog/released-docker-desktop-for-mac-apple-silicon/
I am closing this item. If you have issues please raise them on https://github.com/docker/for-mac/issues

[LinusU]

@nebuk89 Congratulations on the release! Is there any place where we can track the status of getting rid of the Rosetta dependency? I searched the issue tracker where you linked but couldn't find anything. I've also only ever seen references to "some upstream binaries" that are still x86, but never been able to find any more information on which upstream binaries, or if anyone is actively working on it...

Thanks!

[StefanScherer]

@LinusU I don't think we have a public ticket to track, but it's definitely on our list of internal tracked items.
You can read about changes for every Docker Desktop release in the release notes https://docs.docker.com/docker-for-mac/release-notes/ . When we finally get rid of requiring Rosetta it will be shown there, and also the system requirements in our docs https://docs.docker.com/docker-for-mac/install/#system-requirements will be updated.
So, just keep your Docker Desktop installation up to date and follow announcements.
Technically our list is down to a few components, and we're aware of version bumps in several components we use that now have native Apple silicon support like Kubernetes, Docker Engine, Node.js, ...
And before you ask, I don't have an ETA to share 😅

[LinusU]

@StefanScherer thanks for the quick reply, I'll keep watching the release notes then 👍

So, just keep your Docker Desktop installation up to date

(my Docker crashes on startup because of missing Rosetta, so I can't run the updater 😄)

b.t.w. the "Known issues" section doesn't mention that it requires Rosetta anymore (I think it did during the technical preview), which initially led me to believe that the non-preview release didn't require it anymore. I managed to install it, but then Docker crashes on startup.

No big deal though, since I guess that I will just be able to download and run the latest installer as soon as the dependency on Rosetta is dropped 🚀

[tygore587]

b.t.w. the "Known issues" section doesn't mention that it requires Rosetta anymore (I think it did during the technical preview), which initially led me to believe that the non-preview release didn't require it anymore. I managed to install it, but then Docker crashes on startup.

@LinusU https://docs.docker.com/docker-for-mac/apple-silicon/ it is listed as a system requirement in the installation docs.