New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
vmware: minikube iso fails to boot on arm64 #14661
Comments
Seems like it can't find the ISO filesystem, but at least it found the EFI and the kernel
The init (pid 1) is located on the initrd, which is supposed to be loaded by GRUB:
|
For the other hypervisors, there were some arm64 problems with SATA vs SCSI Maybe it is something similar here, vmware using an unexpected CD device ? Either way, probably needs to be looked into upstream (in driver): https://github.com/machine-drivers/docker-machine-driver-vmware |
I doubt docker-machine would be relevant if it fails even without its intervention? it should boot manually before docker-machine has any chance of setting things up correctly either way: VMWare fusion offers 2 cdrom device options: scsi and sata testing the device configuration with ubuntu: perhaps the solution is to build the appropriate modules in buildroot (also for parallels) |
Perhaps, it needs to be cleaned up during the refactoring anyway. Could need contributions |
I think I know what the problem is. looking at the ubuntu boot sequence, immediately following the scsi identification, it registers the cdrom as sr0. looking at https://github.com/kubernetes/minikube/blob/master/deploy/iso/minikube-iso/board/minikube/aarch64/linux_aarch64_defconfig , CONFIG_BLK_DEV_SR isn't set there might be some more scsi setup needed but it is possible that re-enabling this could fix support. while we're at it, CONFIG_VMWARE_PVSCSI=y might help too - if available for aarch64 |
I found a dmesg dump from parallels and can confirm it's also likely the culprit: I am compiling a new iso locally as I write this and will report if it boots on vmware fusion. |
For the libvirt driver (kvm), it was changed here: pkg/drivers/kvm/domain_definition_arm64.go @@ -47,7 +49,7 @@
<devices>
<disk type='file' device='cdrom'>
<source file='{{.ISO}}'/>
- <target dev='hdc' bus='scsi'/>
+ <target dev='sdc' bus='sata'/>
<readonly/>
</disk>
<disk type='file' device='disk'>
|
okay, so I could finally test this. adding the module did in fact cause the machine to detect the cdrom properly and offer it as a root mount (for some reason, boot params needed to be modified for this to work. unsure why it works on other platforms/machines). it now fails on missing filesystem: https://imgur.com/a/zqFQEe6 indeed checking the conf, CONFIG_ISO9660_FS=y is also missing. That's... weird. |
Adding Not sure why they were dropped, maybe @klaases or @sharifelgamal can recall Could be as simple as different defconfig, between the kernel architectures ? i.e. that they were never explicily added, just happened to be there by default. kernel/x86_64_defconfig:CONFIG_BLK_DEV_SD=y kernel/x86_64_defconfig:CONFIG_EXT4_FS=y kernel/aarch64_defconfig:CONFIG_BLK_DEV_SD=y kernel/aarch64_defconfig:CONFIG_EXT2_FS=y |
I will try and retest again tomorrow unless you feel it's right to proceed forth with the requested flag changes for aarch64. summing them up: important for bootCONFIG_BLK_DEV_SR=y should be otherwise prudentCONFIG_VMWARE_BALLOON=m this would not explain why the grub command line had to be manually modified though (the 2nd line removed, added initrd=/boot/initrd and root=/dev/sr0 to the 1st one) |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
What Happened?
In the interest of re-enabling the vmware driver for the mac m1 platform, I have tried (manually) booting the minikube arm iso on vmware fusion. it will not boot.
dmesg screenshot: https://imgur.com/a/SWHiRHX
dmesg output is only available after removing the console=ttyAMA0 from the boot command line
note that this may also be the reason parallels fails to run, can't say as my parallels trial expired
Attach the log file
n/a
Operating System
macOS (Default)
Driver
VMware
The text was updated successfully, but these errors were encountered: