Update Lima to latest so we can try Virtiofs

[ryancurrah]

Problem Description

The bundled version of Lima does not include Virtiosfs Mount mode support.

Proposed Solution

Sync the forked version of Lima and update lima-and-qemu and create a new release of Rancher Desktop. If it doesn't break things.

Additional Information

No response

[ryancurrah]

@jandubois in the meantime is there a way we can get rancher desktop to use a newer version of lima manually? Like copying the latest lima binary to where it is embedded in the rancher desktop install location?

[jandubois]

in the meantime is there a way we can get rancher desktop to use a newer version of lima manually? Like copying the latest lima binary to where it is embedded in the rancher desktop install location?

I will be looking into this in the next few weeks and see if I can provide a replacement tarball that you can use for testing; not sure if it will require a new ISO as well, or not.

If you follow Lima development, you will have noticed that this is still the bleeding edge, and problems continue to be discovered, so it does not seem to be ready for real usage. I will try to find a way how this can be bundled and enabled on an experimental opt-in basis for the next release, but can't promise yet, if this will be possible.

[jandubois]

Just one example of the issues we need to resolve: the VZ code only works if it has been built on a Ventura machine with the latest XCode.

However, that very likely means that the code will not work correctly on older macOS releases. So maybe we need to build multiple instances of Lima, and choose at runtime which version to use.

We still need to support at least Big Sur and Monterey in addition to Ventura (we will drop support for Catalina though, as it is now completely unsupported by Apple).

[ryancurrah]

That is a challenge. It looks like lima is being built on Monterey and switching to a newer xcode version to enable the macOS 13 SDK. https://github.com/lima-vm/lima/blob/master/.github/workflows/release.yml#L19-L28.

And the oldest macOS version that can be used with xcode 14.1 is Monterey 12.5 or later.

Xcode 14.1 requires a Mac running macOS Monterey 12.5 or later.
https://developer.apple.com/documentation/xcode-release-notes/xcode-14_1-release-notes

If you want to build VZ support (Without supporting multiple builds of lima) you could only support Monterey or newer. Big Sur won't be end of life until 12, Nov, 2023.

It looks like a challenge to support older macOS's and all the features lima is coming out with (VZ). That does put a burden on Rancher Desktop. Does Rancher Desktop try to support all the new features in lima by dropping support for older macOS versions or try to support all non-end of life macOS versions even though lima may not.

Our perspective is we are eager to get our hands on the newest improvements in lima.

Looking at some other projects similar to Rancher Desktop...

Colima does not bundle lima it depends on lima via the brew formula. So Colima supports what lima supports.

Finch builds lima on macOS 10.15 for amd64 and macOS 11.11 for arm64.

I am willing to help figure out solution for this and work on some tickets. Not sure if it is wanted or not, just let me know how I can get involved.

[jandubois]

That is a challenge. It looks like lima is being built on Monterey and switching to a newer xcode version to enable the macOS 13 SDK. https://github.com/lima-vm/lima/blob/master/.github/workflows/release.yml#L19-L28.

And the oldest macOS version that can be used with xcode 14.1 is Monterey 12.5 or later.

I noticed. I just did a test build with lima 0.14.2 and the macOS 13 SDK:

https://github.com/rancher-sandbox/lima-and-qemu/actions/runs/3859700464

I will now test this manually, but I only have Catalina and Monterey Intel Macs anymore, so cannot test the VZ bits right now. Will have to do an ARM build manually to test on Ventura.

If you want to build VZ support (Without supporting multiple builds of lima) you could only support Monterey or newer. Big Sur won't be end of life until 12, Nov, 2023.

I'm not opposed to doing multiple builds to support all platforms; it will just take some extra effort. Let's first find out if it is necessary.

It looks like a challenge to support older macOS's and all the features lima is coming out with (VZ). That does put a burden on Rancher Desktop. Does Rancher Desktop try to support all the new features in lima by dropping support for older macOS versions or try to support all non-end of life macOS versions even though lima may not.

Lima does support older macOS versions too; it just builds without VZ support on them:

https://github.com/lima-vm/lima/blob/13fe50e46db259122ea1b919348dfb8a2b0d5764/Makefile#L19-L22

This works fine for Lima because e.g. Homebrew has separate bottles for Big Sur, Monterey, and Ventura.

We only want to have a single build of RD for macOS (well, one for Intel and one for ARM, at least for now), so we may have to bundle multiple versions and decide at runtime which one to use. We'll see. But we definitely want to support all macOS versions that are not yet EOL'ed by Apple.

Our perspective is we are eager to get our hands on the newest improvements in lima.

Looking at some other projects similar to Rancher Desktop...

Colima does not bundle lima it depends on lima via the brew formula. So Colima supports what lima supports.

It does. But if you observed it closely, you can see that colima ran into various issues during the last few weeks and had to back out some changes to make it work again. If you look closely, you'll notice that I fixed one of the issues for them: lima-vm/lima#1243

We try to avoid this instability by waiting until things have settled down a bit, which I hope they have by now (not completely, but maybe good enough for experimental support).

Finch builds lima on macOS 10.15 for amd64 and macOS 11.11 for arm64.

Finch is using the "lima-and-qemu" packaging mechanism from Rancher Desktop.

I am willing to help figure out solution for this and work on some tickets. Not sure if it is wanted or not, just let me know how I can get involved.

Yes, your help is very much appreciated! I'm trying to get at least the lima-and-qemu bits in place, to you can help with the testing. This will not be my main priority for at least another week, but I'll try to work with you to move it along.

I'll update here when I have some testing results; please do the same if you get the chance!

[jandubois]

I'll update here when I have some testing results

For some reason the bundled lima version is not 0.14.2, but a couple commits later; I still have to figure out how that happened. I thought the submodule was pointing the the right commit.

And testing shows that we definitely need 2 versions of Lima, one for Big Sur+, and one for Ventura. I think we can still package a single version of qemu, to save some space (I know, a losing fight, but we have to try).