Mac M1: hyperkit not supported.

[NitichaiSawangsai]

Mac M1

Steps to reproduce the issue:

1. arch -x86_64 brew install hyperkit
2. minikube start --vm=true
3. minikube start --driver=hyperkit

Full output of failed command:

$ minikube start --driver=hyperkit 

😄  minikube v1.21.0 on Darwin 11.4 (arm64)
✨  Using the hyperkit driver based on user configuration

❌  Exiting due to DRV_UNSUPPORTED_OS: The driver 'hyperkit' is not supported on darwin/arm64

[spowelljr]

Hi @NitichaiSawangsai, thanks for reporting your issue!

As shown in the message, minikube does not yet support hyperkit on M1 Macs.

This is something we're hoping to support in the future. For now you can use the Docker driver.

Hope that helps, thanks for using minikube!

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[sharifelgamal]

We're going to work on implementing a qemu driver for a vm driver on arm64 darwin.

[cedricjacobs]

We're going to work on implementing a qemu driver for a vm driver on arm64 darwin.

Any clue when it is planned to be released?

[spowelljr]

Any clue when it is planned to be released?

The current tentative timeline puts it the end of Q2 2022.

[ProbStub]

@spowelljr @sharifelgamal I totally love my M1 but my love for Docker is much more limited and I see the issue remains unassigned. Is there a recommended best practice as a workaround while this is being worked on, e.g. using podman?

[afbjorklund]

You can use lima, as a workaround.

brew install lima

limactl start https://github.com/lima-vm/lima/blob/master/examples/k8s.yaml

It will use ubuntu and containerd.

[spowelljr]

We just released minikube v1.26.0-beta.0 that supports the QEMU driver --driver=qemu2.

https://github.com/kubernetes/minikube/releases/tag/v1.26.0-beta.0

[greenchapter]

We just released minikube v1.26.0-beta.0 that supports the QEMU driver --driver=qemu2.

https://github.com/kubernetes/minikube/releases/tag/v1.26.0-beta.0

Looks on the first sight very good 👏

[tedgoddard]

@spowelljr is it possible to specify the path to qemu? I get

🤷 Exiting due to PROVIDER_QEMU2_NOT_FOUND: The 'qemu2' provider was not found: lookup qemu: open /opt/homebrew/Cellar/qemu: no such file or directory

with an installation to /usr/local from source.

which qemu-system-aarch64
/usr/local/bin/qemu-system-aarch64

(This is also qemu 7, so perhaps I should install qemu 6)

[sharifelgamal]

@spowelljr is it possible to specify the path to qemu? I get

🤷 Exiting due to PROVIDER_QEMU2_NOT_FOUND: The 'qemu2' provider was not found: lookup qemu: open /opt/homebrew/Cellar/qemu: no such file or directory

with an installation to /usr/local from source.

which qemu-system-aarch64
/usr/local/bin/qemu-system-aarch64

(This is also qemu 7, so perhaps I should install qemu 6)

Not yet, this is something we're going to fix before the GA v1.26 release. Right now, we only look for a qemu installation from brew.

#14146 has more details about what's missing in the qemu driver.

[sharifelgamal]

@tedgoddard I merged a PR that should allow you to specify the path of the firmware file (usually edk2-aarch64-code.fd) you want to to use for qemu. We just look for a qemu-system-* binary in your PATH, so that should not be an issue. 

[a1ex-var1amov]

Hi everyone,
Has anybody else faced this with qemu2?

❯ minikube start --driver qemu2 --memory=4Gb
😄  minikube v1.26.0-beta.1 on Darwin 12.4 (arm64)
✨  Using the qemu2 (experimental) driver based on existing profile
❗  You cannot change the memory size for an existing minikube cluster. Please first delete the cluster.
👍  Starting control plane node minikube in cluster minikube
🔄  Restarting existing qemu2 VM for "minikube" ...
OUTPUT:
ERROR: qemu-system-aarch64: Addressing limited to 32 bits, but memory exceeds it by 1073741824 bytes

[sharifelgamal]

yeah, this is a known issue (seen in #14273) that should be fixed at HEAD, and will be fixed when the GA v1.26.0 minikube release is out.

[a1ex-var1amov]

@sharifelgamal thank you so much!

[0xleyth]

I got it to work by doing:

brew install qemu

After the download, I did this:

minikube start --driver qemu --memory=4Gb

[greenchapter]

I got it to work by doing:

brew install qemu

After the download, I did this:

minikube start --driver qemu --memory=4Gb

Thanks that works, but it uses qemu and not Hyperkit 😉

[afbjorklund]

HyperKit is Intel-only, so the alternative to QEMU will be the new Virtualization.framework driver (whenever that is ready)

• Minikube with driver "new Virtualization Framework" on Apple M1 #12826

[greenchapter]

HyperKit is Intel-only, so the alternative to QEMU will be the new Virtualization.framework driver (whenever that is ready)

• Minikube with driver "new Virtualization Framework" on Apple M1 #12826

Ya, you're totally right 👍

[rajjaiswalsaumya]

Can we please document this to the quickstart for mac m1 ?

[dmiyamasu]

Can we please document this to the quickstart for mac m1 ?

Has the quickstart documentation been started for mac m1? I don't see the updates published.

[greenchapter]

Can we please document this to the quickstart for mac m1 ?

Has the quickstart documentation been started for mac m1? I don't see the updates published.

@dmiyamasu and @rajjaiswalsaumya there is a Apple Silicon documentation here → https://minikube.sigs.k8s.io/docs/start/ and also the QEMU startup flags are explained on the drivers page → https://minikube.sigs.k8s.io/docs/drivers/qemu/

I think that will help.

[jgstew]

This document also only mentions hyperkit and doesn't mention using qemu on apple silicon: https://github.com/kubernetes/minikube/blob/master/site/content/en/docs/tutorials/docker_desktop_replacement.md