Test on ARM macOS #42
Comments
|
@AkihiroSuda Is it because of lack of hardware? I may be able to lend a hand and do some testing on mine. |
|
Thanks, testing is highly appreciated. I'm planning to purchase an ARM Mac probably after they release the 2021 model of MacBookPro. |
|
A quick test with a custom-built† QEMU 5.2.0 installed in As was suggested while waiting for ssh, I looked into the boot output in a second term: Alright, everything seems up, let's try in a third term: I'd say it works fine. † followed this, only I added this patch as well |
|
|
|
Thanks! |
|
Just tested on an M1 Mac Mini using what's available in Homebrew: Here's the latter file: |
|
Thanks for testing, but for ARM Mac you have to apply a patch to QEMU https://gist.github.com/nrjdalal/e70249bb5d2e9d844cc203fd11f74c55 If it still does not work, please show serial.log . |
|
Followed the steps on m1 mac installed the qemu patch from https://gist.github.com/nrjdalal/e70249bb5d2e9d844cc203fd11f74c55 when I try to start qemu ~ ❯ qemu-system-aarch64 \ 10:47:49 AM **~ ❯ limactl --debug start** 7s 10:49:31 AM **~ ❯ cat /Users/kiran_chavala/.lima/default/ha.stderr.log** {"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see "/Users/kiran_chavala/.lima/default/serial.log")","time":"2021-08-23T10:49:38+05:30"} |
|
Getting the same error even after ~ ❯ sudo codesign -s - --entitlements entitlements.xml --force /usr/local/bin/qemu-system-aarch64 |
|
@kiranchavala I just followed the same steps from the gist manually, and it worked for me: % qemu-system-aarch64 -accel help
Accelerators supported in QEMU binary:
hvf
tcgThis was on an (almost) freshly install M1 mini: % brew ls
==> Formulae
autoconf gdbm glib m4 ninja pcre pkg-config readline xz
automake gettext libffi mpdecimal openssl@1.1 pixman python@3.9 sqliteThere was one comment in the gist discussion from a person having the same issue: they had python 3.8 installed. Once they moved to Python 3.9 the problem went away. |
|
|
I got the same issue with my Mac on ARM. |
|
Sharing gist note written by @toricls (thank you!) for running Lima on ARM Mac https://gist.github.com/toricls/d3dd0bec7d4c6ddbcf2d25f211e8cd7b |
|
Yay, thank you for sharing my note here @AkihiroSuda! One issue I've faced during my Lima party was something related to bind mount (which is not on the gist note now). Either way I'll create an issue in the |
I had the same thing happen to me (on Intel), and I thought it was because the host directories are mounted read-only into the VM, and then the app wanted to write to it from inside the container. Got distracted with something else; will test it again tomorrow. |
This doesn't seem to be causing the issue (although it would/should have caused the command to fail with an error). I've now mounted the home directory as writable, and the container still hangs. It is supposed to copy a single file out of the container image to the mounted volume: $ docker run -v /Users/jan/tmp/screen/bin:/mnt screen-static-coreos:latest cp -v /home/core/screen-4.8.0/screen /mnt |
|
Hi all, May I know if anyone tried intel-on-arm on M1 Mac? Should I use x86_64 brew-ed qemu instead, or? Thanks! $ limactl start x
x
? Creating an instance "x" Open an editor to override the configuration
INFO[0204] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.11.1/nerdctl-full-0.11.1-linux-amd64.tar.gz" (sha256:ce7a6e119b03c3fb8ded3d46d929962fd17417bea1d5bbc07e0fce49494d8a09)
INFO[0204] Using cache "/Users/weyl/Library/Caches/lima/download/by-url-sha256/3304d173f1e1824e5be6cf84bf2f78825cf0db416c4c975dbb2458776942629e/data"
INFO[0205] Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-amd64.img"
INFO[0205] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img"
INFO[0205] Using cache "/Users/weyl/Library/Caches/lima/download/by-url-sha256/e1fed960ebd29619676c7ab7535bc83f7fb2ad71739edb6fde4e17bce0b61a47/data"
INFO[0205] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/weyl/.lima/x/serial.log")
INFO[0205] SSH Local Port: 62022
INFO[0205] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0293] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0380] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0467] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0553] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0640] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0727] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
FATA[0805] did not receive an event with the "running" status
$ tail -f *.log
==> ha.stderr.log <==
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EDX.spec-ctrl [bit 26]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.fma [bit 12]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.x2apic [bit 21]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.avx [bit 28]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.f16c [bit 29]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.avx2 [bit 5]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EBX.invpcid [bit 10]","time":"2021-09-06T16:27:19+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.07H:EDX.spec-ctrl [bit 26]","time":"2021-09-06T16:27:19+08:00"}
==> ha.stdout.log <==
{"time":"2021-09-06T16:27:19.070538+08:00","status":{"sshLocalPort":62022}}
==> serial.log <==
==> ha.stderr.log <==
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:28:36+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:28:46+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:28:46+08:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:30:03+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:30:13+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:30:13+08:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:31:30+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:31:40+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:31:40+08:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\n\": exit status 255","time":"2021-09-06T16:32:57+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 4: \"ssh\"","time":"2021-09-06T16:33:07+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2021-09-06T16:33:07+08:00"}
$ qemu-system-x86_64 -accel help
Accelerators supported in QEMU binary:
tcg |
I don't know; how long did you wait? I once tried running the ARM version on an Intel Mac, and thought it didn't work, but I just didn't wait long enough; it took 30min to boot the VM. Maybe I've done something wrong, but I think emulating the whole OS with a software-level emulator is just too slow. We can run just containers from a different architecture using qemu-user mode together with |
|
Thank you @jandubois! I come to know software emulation of the VM level should be unusably slow and costly. I am waiting for And it was 13 min waiting by the time I collected the log, maybe to wait longer will have the OS booted up. Thank you so much! You rock!!! |
|
Updating installation steps for ARM ( Can anyone confirm whether this works with Lima? |
|
@AkihiroSuda nixpkgs user so can't help here, although FYI there's a bit of work at nixpkgs to improve entitlements and signatures for qemu on aarch64-darwin, so it should be as easy as e.g |
|
Just reporting I've been working with this for the past two days with great success. After creating a vm, I run |
Thanks for this tip! I've added this as a provision script in lima config: |
I have installed lima use brew,but after installed i follow the readme.md ,uninstall the qemu 0.6.1 brew uninstall --ignore-dependencies qemureinstall qemu with patch brew install simnalamburt/x/qemu-hvfverify qemu version shows after all these , lima works fine |
@AkihiroSuda Can highly recommend Scaleway for testing on Apple Silicon (Mac Mini) in the cloud... |
|
Upstream brew patched qemu, so it is no longer necessary to use the tap. Homebrew/homebrew-core@5e8eb54#diff-5fe005c786b1e905434aaa353962cc6a58b0a6431e7f9a785076f0ad5f277a3d |
Just updated my installation guide to use Homebrew to install QEMU and Lima. It's dead simple now 😍 |
|
Happy to contribute to testing on M1, I'm on Monterey beta 10/MBA M1 16GB. FWIW default image works (ubuntu) but fedora does not nor centos when I created a centos.yaml (pretty much the same as the fedora.yaml but using centos cloud image URLs and sha). It looks like the initial difficulty is |
|
Thanks for testing, ARM edition of Fedora is now supported in the master #329 Will be included in v0.7.2 (probably next week) |
|
The fedora/centos issues were related to Since CentOS (CentOS Linux) is going end-of-life soon (2021-12-31), it probably needs replacing with "Stream" (CentOS Stream). |
|
i have tried on my mac M1. it seems the data in shared media is always lost when stopped and restarted on the vm container and I have problems with the network. vm can't connect to the internet like doing a clone repository action on github.com |
|
Smooth sailing so far on M1 Pro, though that's not surprising because I wouldn't expect silicon-level breaking changes. |
|
It's been working pretty well for me however I'm receiving an error when I stop a container and try to run it again on the same port: If I then try a third time the container starts but the port still hangs. If I stop and start with limactl it works again. UPDATE: I see same behavior on intel macbook, so looks like it is not an ARM specific issue |
|
This is due to containerd/nerdctl#458 . |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Sorry, moved the conversation about architecture-native binaries and images to #406 |
|
I could offer some testing (currently Air M1, soon probably another M1 Pro). But I'm unsure what tests I can contribute. I did some very simple Today I tried running a |
|
Hi, trying to get working on an M1 Mac - Big Sur 11.4. I've been stuck at this for a while now: Is there something that I am doing wrong? I installed lima via |
There might be more clues in that "serial.log" file |
I'm not sure if this applies to your situation, but in my case this error was due to my local HTTP proxy configuration in macOS. After enabling video output to view the logs on init, I noticed that when installing After this the default ubuntu image works flawlessly on the M1 for me. |
There were various issues with environment variables in general, and proxy settings in particular in earlier lima releases, but I'm not aware of any remaining issues in v0.7.3 or later. If the latest lima release does not automatically inherit proxy setting, then please file a separate Github issue with the details, so we can investigate. @AkihiroSuda I'm tempted to close this bug, as it seems to just collect all manner of feedback related to M1 support instead of being specific to a single issue. I.e. I cannot tell what actions would be required to resolve it. |
|
Sure, closing this issue. |
and others
No description provided.
The text was updated successfully, but these errors were encountered: