Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fallback to installing AMD64 plugins on Darwin #491

Merged

Conversation

marckhouzam
Copy link
Contributor

@marckhouzam marckhouzam commented Sep 21, 2023

What this PR does / why we need it

With this PR it is now possible to fully use a locally built CLI on Mac ARM64 machines

Although the CLI is available for Darwin ARM64, the vast majority of plugins are not, and therefore an ARM64 CLI will not be able to install most plugins.

Thanks to Apple's Rosetta emulator available on Darwin ARM64 machines, it is possible to run AMD64 binaries. This is currently how the CLI and its plugins are being run.

This commit teaches a Darwin ARM64 CLI that if a plugin is not available for Darwin ARM64, it should instead install the Darwin AMD64 version. With this approach, it now become possible to use a Darwin ARM64 CLI to its full potential.

Note that this approach cannot be used for Linux as there is no standard emulator for AMD64 binaries. For Linux, plugins will need to be published for ARM64.

Unit tests have been added for this new feature.

Which issue(s) this PR fixes

Part of #357

Describe testing done for PR

# Reset the CLI
$ rm ~/.cache/tanzu/catalog.yaml
$ rm ~/.config/tanzu/config*
$ rm ~/.cache/tanzu/plugin_inventory

# Build an ARM64 CLI on Darwin
$ make build
build darwin-arm64 CLI with version: v1.1.0-dev
mkdir -p bin
cp /Users/kmarc/git/tanzu-cli/artifacts/darwin/arm64/cli/core/v1.1.0-dev/tanzu-cli-darwin_arm64 ./bin/tanzu

$ tz ceip set false
$ tz config eula accept
[ok] Marking agreement as accepted.

# Install a plugin that is not available in ARM64
$ tz plugin install telemetry --target global
[i] Reading plugin inventory for "projects.registry.vmware.com/tanzu_cli/plugins/plugin-inventory:latest", this will take a few seconds.
[i] Installing plugin 'telemetry:v1.1.0' with target 'global'
[i] Plugin binary for 'telemetry:v1.1.0' found in cache
[ok] successfully installed 'telemetry' plugin

# Confirm that the installed plugin binary is AMD64
$ tz plugin describe telemetry
  NAME       VERSION  STATUS     TARGET  DESCRIPTION                                                 INSTALLATIONPATH
  telemetry  v1.1.0   installed  global  configure cluster-wide settings for vmware tanzu telemetry  /Users/kmarc/Library/Application
                                                                                                     Support/tanzu-cli/telemetry/v1.1.0_a93519154dc71368d738aec09f9bc964a0c3bcd02a856106ccaddaaf31426636_global
$ file ~/Library/Application\ Support/tanzu-cli/telemetry/v1.1.0_a93519154dc71368d738aec09f9bc964a0c3bcd02a856106ccaddaaf31426636_global
/Users/kmarc/Library/Application Support/tanzu-cli/telemetry/v1.1.0_a93519154dc71368d738aec09f9bc964a0c3bcd02a856106ccaddaaf31426636_global: Mach-O 64-bit executable x86_64

# Now install a plugin that is available in ARM64
$ tz plugin install builder
[i] Installing plugin 'builder:v1.0.0' with target 'global'
[i] Plugin binary for 'builder:v1.0.0' found in cache
[ok] successfully installed 'builder' plugin

# Confirm that the installed plugin binary is ARM64
$ tz plugin describe builder
  NAME     VERSION  STATUS     TARGET  DESCRIPTION             INSTALLATIONPATH
  builder  v1.0.0   installed  global  Build Tanzu components  /Users/kmarc/Library/Application
                                                               Support/tanzu-cli/builder/v1.0.0_9e15f81e9a1dc5655aca06562922a48525231072b30033a187aef78ff56d3f74_global
$ file ~/Library/Application\ Support/tanzu-cli/builder/v1.0.0_9e15f81e9a1dc5655aca06562922a48525231072b30033a187aef78ff56d3f74_global
/Users/kmarc/Library/Application Support/tanzu-cli/builder/v1.0.0_9e15f81e9a1dc5655aca06562922a48525231072b30033a187aef78ff56d3f74_global: Mach-O 64-bit executable arm64

# Create a context for TMC.  None of the plugins are available in ARM64
# but with this PR, the installation of those plugins will now work
$ tz context create tmc2 --endpoint unstable.tmc-dev.cloud.vmware.com --staging

[i] If you don't have an API token, visit the VMware Cloud Services console, select your organization, and create an API token with the TMC service roles:
  https://console.cloud.vmware.com/csp/gateway/portal/#/user/tokens

? API Token ****************************************************************

[ok] successfully created a TMC context
[i] Checking for required plugins...
[i] Installing plugin 'secret:v0.1.4' with target 'mission-control'
[i] Installing plugin 'clustergroup:v0.1.4' with target 'mission-control'
[i] Installing plugin 'account:v0.1.4' with target 'mission-control'
[i] Installing plugin 'integration:v0.1.4' with target 'mission-control'
[i] Installing plugin 'aks-cluster:v0.1.7' with target 'mission-control'
[i] Installing plugin 'provider-eks-cluster:v0.1.4' with target 'mission-control'
[i] Installing plugin 'apply:v0.3.1' with target 'mission-control'
[i] Installing plugin 'setting:v0.2.2' with target 'mission-control'
[i] Installing plugin 'tanzupackage:v0.2.3' with target 'mission-control'
[i] Installing plugin 'continuousdelivery:v0.1.4' with target 'mission-control'
[i] Installing plugin 'ekscluster:v0.1.4' with target 'mission-control'
[i] Installing plugin 'helm:v0.1.4' with target 'mission-control'
[i] Installing plugin 'inspection:v0.1.4' with target 'mission-control'
[i] Installing plugin 'provider-aks-cluster:v0.1.4' with target 'mission-control'
[i] Installing plugin 'iam:v0.1.4' with target 'mission-control'
[i] Installing plugin 'audit:v0.1.4' with target 'mission-control'
[i] Installing plugin 'events:v0.1.4' with target 'mission-control'
[i] Installing plugin 'workspace:v0.1.6' with target 'mission-control'
[i] Installing plugin 'data-protection:v0.1.4' with target 'mission-control'
[i] Installing plugin 'policy:v0.1.4' with target 'mission-control'
[i] Installing plugin 'management-cluster:v0.2.4' with target 'mission-control'
[i] Installing plugin 'cluster:v0.1.6' with target 'mission-control'
[i] Plugin binary for 'cluster:v0.1.6' found in cache
[i] Installing plugin 'agentartifacts:v0.1.4' with target 'mission-control'
[i] Successfully installed all required plugins

# Check that these plugin binaries are AMD64
$ file ~/Library/Application\ Support/tanzu-cli/agentartifacts/v0.1.4_8f6cbbbd670ad0c32bc39adc37835e6d4dac28b66230af4517aa6c3ff83b3c3c_mission-control
/Users/kmarc/Library/Application Support/tanzu-cli/agentartifacts/v0.1.4_8f6cbbbd670ad0c32bc39adc37835e6d4dac28b66230af4517aa6c3ff83b3c3c_mission-control: Mach-O 64-bit executable x86_64
$ file ~/Library/Application\ Support/tanzu-cli/events/v0.1.4_e93c2eb67ee0cc21a0997cd116b66d81cc40064c121483daff530d31dd45cd9f_mission-control
/Users/kmarc/Library/Application Support/tanzu-cli/events/v0.1.4_e93c2eb67ee0cc21a0997cd116b66d81cc40064c121483daff530d31dd45cd9f_mission-control: Mach-O 64-bit executable x86_64

Release note

Allow fully using an ARM64 CLI on Mac (Darwin) by having the CLI install plugins for AMD64 when an ARM64 version of the plugin is not available. 

Additional information

Special notes for your reviewer

Implementation approach
The implementation approach chosen is to replace the use of runtime.GOOS/GOARCH with our own version cli.GOOS/GOARCH everywhere in the code. The new variable are set to the same value as the runtime ones. However, it now becomes possible to programatically make the CLI believe it is running on a different OS/ARCH (for this PR, we change the ARCH to AMD64 to install plugins not available for ARM64 on Darwin).

An alternative approach would have been to add arguments to many different function to specify what ARCH should be used. This second approach would have required a bit more changes.

However, the real reason I opted for the first approach is that it also allows to write unit tests for this feature, even if those tests are run on Linux, or AMD64, by using the cli.GOOS/GOARCH to make the CLI believe it is running on Darwin ARM64 for the test in question.

No E2E tests

To be able to use this feature, one must run on a Darwin ARM64 machine. Therefore to be able to exercise the feature in E2E tests, those tests would need to run on darwin_arm64, which is not something offered out-of-the-box for Github actions. Here are the options I can see:

  1. implement an e2e test which will only test the feature when run manually by someone on their darwin_arm64 machine
    • inconsistent testing since requires to run it manually
  2. deploy a self-hosted runner on a darwin_arm64 machine and have github run our e2e tests on it
    • does not seem possible to run a MacOS VM on Azure. Not sure of other options.
  3. don't implement an e2e test for this feature and rely on the existing unit test
    • should be a sufficient indication that the feature is functional

I prefer option 3 but am open to other opinions or suggestions.

Other aspect

A aspect that I did NOT implement is the following: when running a Darwin AMD64 CLI on a Darwin ARM64 machine, we could install ARM64 plugins when available; now, and with this PR, an Darwin AMD64 will only install AMD64 plugins.

I opted against this because with this PR, there should be no reason to run an AMD64 CLI on a Darwin ARM64 machine. Therefore we should direct users to use the ARM64 CLI. Same goes for plugin devs. Note that 'brew install' will install the ARM64 CLI starting with the v1.1 version when on ARM64 Macs.

Besides, even if we did implement this extra "feature", a plugin dev would need to install the v1.1 CLI to benefit from it, which takes us back to: might as well install the ARM64 version of the CLI 1.1.

Although the CLI is available for Darwin ARM64, the vast majority of
plugins are not, and therefore an ARM64 CLI will not be able to install
most plugins.

Thanks to Apple's Rosetta emulator available on Darwin ARM64 machines,
it is possible to run AMD64 binaries.  This is currently how the CLI
and its plugins are being run.

This commit teaches a Darwin ARM64 CLI that if a plugin is not available
for Darwin ARM64, it should instead install the Darwin AMD64 version.
With this approach, it now become possible to use a Darwin ARM64 CLI to
its full potential.

Note that this approach cannot be used for Linux as there is no standard
emulator for AMD64 binaries.  For Linux, plugins will need to be
published for ARM64.

Unit tests have been added for this new feature.

Signed-off-by: Marc Khouzam <kmarc@vmware.com>
Copy link
Contributor

@anujc25 anujc25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat 👏 . Thanks for this change. I could not test this manually as I do not have an M1/M2 Mac but thanks for the detailed PR description.

LGTM.

@marckhouzam marckhouzam merged commit 8b7c128 into vmware-tanzu:main Sep 26, 2023
4 checks passed
@marckhouzam marckhouzam deleted the feat/arm64FallbackToAmd64 branch September 26, 2023 23:27
@marckhouzam
Copy link
Contributor Author

@scothis this may interest you

@marckhouzam marckhouzam added this to the 1.1.0 milestone Oct 20, 2023
@marckhouzam marckhouzam added the docs-impact issues with documentation impact label Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-not-required docs-impact issues with documentation impact
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants