exp(feat): enable Virtualization.framework and Rosetta #282

pendo324 · 2023-03-10T23:19:43Z

Issue #, if available: Closes #218

Description of changes:
Adds support for Lima's Virtualization.framework and Rosetta features, through the use of new finch.yaml configuration options (vmType and rosetta). vmType also sets the mountType to virtiofs, since that is only available when using Virtualization.framework.

To support this, a few things needed to be changed on our side:

Disk migration. Although feat: mount additional disks when using vz lima-vm/lima#1405 (which adds persistent disk support to vmType: vz in Lima) will auto-convert persistent disks from QCOW2 to RAW when they are attempted to be used with vmType: vz, because of the way our disks are persisted with symlinks, we also have to do this
Move some hardcoded lima config from finch.yaml to be programatically toggled in pkg/config/lima_config_applier
- This allows things like the qemu user mode scripts to be installed only when they are needed. Installing them all the time, and then trying to use Rosetta as a binformat_misc handler causes conflicts
- This also opens up possibilities of future customization based on Finch's config values

Currently, because lima-vm/lima#1405 is not merged yet, this PR references a specific branch of my own finch-core repo which includes the Lima change already. It also edits the Makefile to do a build of Lima from the submodule directly, and overwrite the Lima downloaded from the archive. These changes will be removed once the Lima change is merged upstream.

Testing done:

unit tests
e2e tests
local testing on Intel and Apple Silicon machines
I've reviewed the guidance in CONTRIBUTING.md

License Acceptance

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

BEGIN_COMMIT_OVERRIDE

exp(feat): enable Virtualization.framework and Rosetta (#282)

END_COMMIT_OVERRIDE

pendo324 · 2023-03-10T23:52:47Z

All of the lint failures finally helped me fix my lint-on-save. Apparently I had to do a >Go: install/update tools in VSCode. It was throwing a very cryptic error before that golangci/golangci-lint#3533.

.gitmodules

Makefile

ningziwen · 2023-03-15T06:00:37Z

Need e2e tests and README.md change

pendo324 · 2023-03-17T23:54:02Z

Need e2e tests and README.md change

Added some basic e2e tests for the two new scenarios and updated the README.

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

…osetta Signed-off-by: Justin Alvarez <alvajus@amazon.com>

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

increase test timeout Signed-off-by: Justin Alvarez <alvajus@amazon.com>

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pkg/config/config.go

finch.yaml

e2e/vm/virtualization_framework_rosetta_test.go

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pendo324 · 2023-03-28T17:05:48Z

Merging, and will investigate the Rosetta test failure as a separate issue

Issue #, if available: *Description of changes:* - Add a new "expr" PR type to track experimental features. An "experimental" feature is something that can be used, but is not guaranteed to be stable (such as the changes made by #282). *Testing done:* - N/A - [x] I've reviewed the guidance in CONTRIBUTING.md #### License Acceptance By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. --------- Signed-off-by: Justin Alvarez <alvajus@amazon.com>

🤖 I have created a release *beep* *boop* --- ## [0.5.0](v0.4.1...v0.5.0) (2023-03-29) ### Features * adds support bundles ([#210](#210)) ([cc6be65](cc6be65)) ### Bug Fixes * Correctly set and pick up environment variables ([#315](#315)) ([05227ce](05227ce)) * print debug logs after newline ([#273](#273)) ([8faa7de](8faa7de)) * print debug logs when lima disk command fails ([#270](#270)) ([78a3f50](78a3f50)) ### Experimental * **feat:** enable Virtualization.framework and Rosetta ([#282](#282)) ([fd3bf19](fd3bf19)) ### Build System or External Dependencies * clean up finch-core _output directory in clean target ([#290](#290)) ([4684a95](4684a95)) * **deps:** bump github.com/onsi/ginkgo/v2 from 2.8.4 to 2.9.0 ([#265](#265)) ([7e2d49e](7e2d49e)) * **deps:** bump github.com/onsi/ginkgo/v2 from 2.9.0 to 2.9.1 ([#285](#285)) ([d741a03](d741a03)) * **deps:** Bump github.com/onsi/gomega from 1.27.3 to 1.27.5 ([#312](#312)) ([e078234](e078234)) * **deps:** bump github.com/runfinch/common-tests from 0.6.1 to 0.6.2 ([#300](#300)) ([dd626a0](dd626a0)) * **deps:** bump github.com/spf13/afero from 1.9.4 to 1.9.5 ([#263](#263)) ([a0e277f](a0e277f)) * **deps:** bump golang.org/x/crypto from 0.6.0 to 0.7.0 ([#264](#264)) ([ec1c07f](ec1c07f)) * **deps:** bump golang.org/x/tools from 0.6.0 to 0.7.0 ([#268](#268)) ([8072e39](8072e39)) * **deps:** bump k8s.io/apimachinery from 0.26.2 to 0.26.3 ([#306](#306)) ([fe392cb](fe392cb)) * **deps:** Bump lima version ([#302](#302)) ([0269743](0269743)) * **deps:** Bump submodules ([#281](#281)) ([d4fd1f6](d4fd1f6)) * **deps:** Bump submodules ([#304](#304)) ([b38af9f](b38af9f)) * **deps:** Bump submodules ([#307](#307)) ([1a914ec](1a914ec)) * **deps:** Bump submodules ([#319](#319)) ([e671224](e671224)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

pendo324 added the enhancement New feature or request label Mar 10, 2023

pendo324 requested a review from ningziwen March 10, 2023 23:19

pendo324 self-assigned this Mar 10, 2023

pendo324 commented Mar 11, 2023

View reviewed changes

.gitmodules Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

pendo324 added 7 commits March 24, 2023 14:26

feat: enable Virtualization.framework and Rosetta

b0da6e3

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

Fix comments in pkg/system

8fb9eb2

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

Fix linting errors in pkg/disk

9ec9600

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

Fix linting errors in pkg/config

6946f61

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pkg/disk: fix formatting

a3aa3af

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pkg/config: fix more linting warnings

1a2001a

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

update README, add basic tests when using Virtualzation.framework / R…

becc675

…osetta Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pendo324 force-pushed the vz-rosetta-support branch from d0dd86c to 5ec0d38 Compare March 24, 2023 18:45

update submodule and makefile

8986927

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pendo324 force-pushed the vz-rosetta-support branch from 5ec0d38 to 8986927 Compare March 24, 2023 18:45

pendo324 closed this Mar 24, 2023

pendo324 reopened this Mar 24, 2023

reset makefile / submodules

d1f8ada

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pendo324 marked this pull request as ready for review March 27, 2023 17:33

pendo324 added 7 commits March 27, 2023 15:28

fix tests

811e6ae

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

Fix linting error

8b254b1

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

update minimum macOS version for VZ to 13

2db4ff0

increase test timeout Signed-off-by: Justin Alvarez <alvajus@amazon.com>

update tests

5fd4995

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

fix over zealous refactoring of test

c26e6df

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

relaunch vm after reset

0417edf

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

Merge branch 'main' into vz-rosetta-support

d1684be

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

ningziwen reviewed Mar 27, 2023

View reviewed changes

pkg/config/config.go Outdated Show resolved Hide resolved

finch.yaml Show resolved Hide resolved

finch.yaml Show resolved Hide resolved

e2e/vm/virtualization_framework_rosetta_test.go Show resolved Hide resolved

Add step to install Rosetta 2

5d3f16a

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pendo324 added 2 commits March 27, 2023 20:00

Fix docs

4da7f1a

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

ignore exit code for Rosetta install

15150b3

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

ningziwen previously approved these changes Mar 28, 2023

View reviewed changes

increase test timeouts

7727e73

Signed-off-by: Justin Alvarez <alvajus@amazon.com>

pendo324 dismissed ningziwen’s stale review via 7727e73 March 28, 2023 15:56

ningziwen approved these changes Mar 28, 2023

View reviewed changes

pendo324 enabled auto-merge (squash) March 28, 2023 17:06

pendo324 disabled auto-merge March 28, 2023 17:06

pendo324 merged commit fd3bf19 into runfinch:main Mar 28, 2023

github-actions bot mentioned this pull request Mar 28, 2023

chore(main): release 0.5.0 #266

Merged

This was referenced Mar 28, 2023

Create new e2e test entry points for Virtualization.framework / Rosetta #321

Open

Fix test failure when running Rosetta tests #322

Closed

ci: add "exp" PR type #323

Merged

pendo324 changed the title ~~feat: enable Virtualization.framework and Rosetta~~ exp(feat): enable Virtualization.framework and Rosetta Mar 28, 2023

pendo324 changed the title ~~exp(feat): enable Virtualization.framework and Rosetta~~ exp(feat): enable Virtualization.framework and Rosetta Mar 28, 2023

github-actions bot mentioned this pull request Mar 30, 2023

chore(main): release 0.6.0 #334

Closed

pendo324 mentioned this pull request Apr 4, 2023

Cross architecture support for dynamic linker (ld) #198

Closed

weikequ mentioned this pull request Apr 13, 2023

Apple's virtualization framework could be used on EC2 instances to enable 2x the number of available GitHub runners runfinch/infrastructure#139

Closed

github-actions bot mentioned this pull request Jul 18, 2023

chore(main): release 0.8.0 #484

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

exp(feat): enable Virtualization.framework and Rosetta #282

exp(feat): enable Virtualization.framework and Rosetta #282

pendo324 commented Mar 10, 2023 •

edited

Loading

pendo324 commented Mar 10, 2023

ningziwen commented Mar 15, 2023

pendo324 commented Mar 17, 2023

pendo324 commented Mar 28, 2023

exp(feat): enable Virtualization.framework and Rosetta #282

exp(feat): enable Virtualization.framework and Rosetta #282

Conversation

pendo324 commented Mar 10, 2023 • edited Loading

License Acceptance

pendo324 commented Mar 10, 2023

ningziwen commented Mar 15, 2023

pendo324 commented Mar 17, 2023

pendo324 commented Mar 28, 2023

pendo324 commented Mar 10, 2023 •

edited

Loading