Create an ARM based build for the pulumi cli (and plugins) #4868

jaxxstorm · 2020-06-22T18:37:46Z

The prerequisite work is to get our projects to go1.16 so that macOS arm builds are supported

The work required here is as follows:

Update tf providers with arm builds
Update azure-native with arm build
Update pulumi-kubernetes with arm build
Update pulumi/pulumi with arm build
Update pulumi/awsx with arm build
Update pulumi/eks with arm build
Update plugin acquisition to detect arm in the downloads
Update get.pulumi.com to allow arm downloads of the CLI
Add some nightly tests to ensure that we can install and use the arm builds

justinvp · 2020-06-22T19:26:01Z

https://www.apple.com/newsroom/2020/06/apple-announces-mac-transition-to-apple-silicon/

lukehoban · 2020-06-22T19:34:38Z

Just a note - we will want ARM builds across all supported OS’s. Linux builds will also be valuable to support using Pulumi on cloud-based ARM servers such as AWS Graviton.

lukehoban · 2020-06-22T19:36:05Z

Also - this will require not just CLI, but also all plugins (both resource provider plugins but also language host plugins).

jaxxstorm · 2020-09-11T02:51:39Z

related: #5244

lukehoban · 2020-09-28T23:20:40Z

Per #5466 - this should also include ARM Docker images.

justinvp · 2020-11-10T18:53:37Z

ARM-based MacBook Pro 13", MacBook Air, and Mac mini available starting 11/17: https://www.apple.com/apple-events/november-2020/

jaxxstorm · 2020-11-11T17:04:01Z

I had a try at building the ARM binaries yesterday, the PR is here:

#5729

The binary worked without issue. I also built an AWS provider ARM binary which worked without any issues either.

I tried out a simple Pulumi program on an AWS graviton machine. It worked successfully with the Go SDK

What I ran into was:

the language SDKs need to have GRPC binaries available
- Python didn't work (even though it's allegedly meant to be available, pip install failed)
- Go did work
plugin acquisition doesn't recognise the machine's chipset yet, so plugin acquisition failed on Python

lukehoban · 2020-11-11T17:20:12Z

Great!

plugin acquisition doesn't recognise the machine's chipset yet, so plugin acquisition failed on Python

What exactly was the error? Was this due to

pulumi/sdk/go/common/workspace/plugins.go

Line 190 in 186e2f0

    
           return nil, -1, errors.Errorf("unsupported plugin architecture: %s", runtime.GOARCH)

Python didn't work (even though it's allegedly meant to be available, pip install failed)

What exactly was the error?

roderik · 2020-12-28T19:45:48Z

There is a Go prerelease supporting the M1 but it is not installable via Homebrew
Python 3.9 is available in homebrew ARM so easy to use

I'm working on an M1 with the typescript SDK's (and the x86 version of pulumi via rosetta) which works fine, so if I can help test an ARM version, let me know

kherings · 2021-01-08T13:43:30Z

Hi everyone, I ran into the problem described above by @jaxxstorm

pulumi plugin install resource aws v3.22.1
[resource plugin aws-3.22.1] installing
error: [resource plugin aws-3.22.1] downloading from : unsupported plugin architecture: arm64

is there any approach to work around this? (M1 Mac)

stack72 · 2021-01-21T14:27:07Z

Unfortunately, this is blocked until Go 1.16 has been released goreleaser/goreleaser#1952

roderik · 2021-01-22T09:26:34Z

is there any approach to work around this? (M1 Mac)

I install 2 versions of homebrew, the ARM one (which uses /opt/homebrew) and the x86 one which uses /usr/local. See point 4 in https://medium.com/better-programming/5-things-i-have-learned-when-using-the-m1-chip-macbook-air-a77f93c50381

I do not use 2 terminals but I aliased this alias ibrew='arch -x86_64 /usr/local/bin/brew'

Install pulumi from ibrew and you get the x86 version that works perfectly over Rosetta
It also installs with arm brew, but then you get the plugin errors you mention here

jamesla · 2021-03-04T05:26:30Z

I managed to get the aws plugins working by compiling myself with the help of @stack72.

For anybody like me that can't wait the steps I followed were:

clone pulumi-aws repository
add arm64 as an architecture in .goreleaser.prerelease.yml
comment out pulumi plugin install resource tls 2.0.0 and pulumi plugin install resource github 1.0.0 in Makefile (expecting this to bite me at some point - but works fine for now.
make a local git commit and then tag it with a valid semver version
goreleaser build -f .goreleaser.prerelease.yml
copy dist/pulumi-aws_darwin_arm64/pulumi-resource-aws to somewhere on $PATH
now you should be able to run pulumi up

Seems to work fine for me although my stacks aren't very complex.

stack72 · 2021-03-04T22:42:42Z

Quick update on this work:

pulumi-azure-native & pulumi-kubernetes are now on go1.16 so that paves the way for arm64 builds for those
pulumi/pulumi has a PR open Upgrading Go to 1.16.x #6470
pulumi tf-based providers also will have PRs open for go1.16 tomorrow (Friday March 4)

Related: #4868 Also adds the arm64 build and deployment steps via goreleaser

stack72 · 2021-03-10T15:37:36Z

Ok further update:

arm64 configs have been added to pulumi-azure-native, pulumi, pulumi-eks & pulumi-kubernetes
arm64 configs have been added to all tf based providers e.g. pulumi-aws, pulumi-gcp
PR open to allow pulumi plugin install to install arm64 plugins

Still remaining: add the work for arm64 downloads to get.pulumi.com

roderik · 2021-03-10T15:49:53Z

FYI: kubectl is not arm ready until this makes it into a release kubernetes/kubernetes#97743 (so at least the kubernetes and eks providers will be impacted)

roderik · 2021-03-10T15:51:18Z

Oh, and google-cloud-sdk also does not install under arm.

The current list I need to install under amd64 is

kubectx
kubernetes-cli
pulumi
google-cloud-sdk

and in case if anyone wonders how I do this

Install 2 versions of homebrew: https://github.com/roderik/dotfiles/blob/main/install#L26-L39
Use an ibrew alias for the amd64 version: https://github.com/roderik/dotfiles/blob/main/zsh/.zshrc#L70
Make sure everything is in your path: https://github.com/roderik/dotfiles/blob/main/zsh/.zshrc#L3

then brew install will do arm versions and ibrew install does the amd64 versions, preferring the arm versions if there are conflicts

Related: #4868 Also adds the arm64 build and deployment steps via goreleaser

stack72 · 2021-03-12T12:06:15Z

Hi all

Pleased to say that this work will be available Wednesday 17th March as part of Pulumi v2.23.0

Paul

stack72 · 2021-03-17T20:47:48Z

Friends, I am happy to say that the latest CLI (v2.23.0) and the latest provider binaries are available with arm64 packages on macOS

Please let us know if there are any issues

Paul

barakcoh · 2021-03-18T08:57:32Z

@stack72 I've successfully upgraded the CLI to 2.23.0 on a macbook m1 but I'm unable to use it because the AWS provider won't install

❯ brew upgrade pulumi
Warning: pulumi 2.23.0 already installed
❯ pulumi destroy -s unityscm/apps/dev-pr241

Previewing destroy (unityscm/dev-pr241)

View Live: https://app.pulumi.com/unityscm/apps/dev-pr241/previews/57a0d909-7272-41ea-9c32-dea336f0bf9a


warning: A new version of Pulumi is available. To upgrade from version '0.0.0' to '2.23.0', run
   $ brew upgrade pulumi
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
error: could not load plugin for aws provider 'urn:pulumi:dev-pr241::apps::pulumi:providers:aws::default_3_26_1': no resource plugin 'aws-v3.26.1' found in the workspace or on your $PATH, install the plugin using `pulumi plugin install resource aws v3.26.1`


❯ pulumi plugin install resource aws v3.26.1
[resource plugin aws-3.26.1] installing
warning: A new version of Pulumi is available. To upgrade from version '0.0.0' to '2.23.0', run
   $ brew upgrade pulumi
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
error: [resource plugin aws-3.26.1] downloading from : 403 HTTP error fetching plugin from https://get.pulumi.com/releases/plugins/pulumi-resource-aws-v3.26.1-darwin-arm64.tar.gz

stack72 · 2021-03-18T10:58:00Z

Hey @barakcoh

So only the latest versions of the plug-in are built with arm so you'd have to upgrade you code to use Pulumi-aws v3.33.0 before there is an arm binary to use

Sorry for me not being clear on that point

Paul

Adam-Burke · 2021-03-22T12:36:51Z

Hey @barakcoh

So only the latest versions of the plug-in are built with arm so you'd have to upgrade you code to use Pulumi-aws v3.33.0 before there is an arm binary to use

Hi @stack72,

I have the same issue as @barakcoh, after updating to the following

    "@pulumi/aws": "^3.34.1",
    "@pulumi/awsx": "^0.25.0",
    "@pulumi/pulumi": "^2.23.1"

I get the following error

error: could not load plugin for aws provider 'urn:pulumi:staging::env::pulumi:providers:aws::default_2_13_1': no resource plugin 'aws-v2.13.1' found in the workspace or on your $PATH, install the plugin using 'pulumi plugin install resource aws v2.13.1'

Then if I try to install that

error: [resource plugin aws-2.13.1] downloading from : 403 HTTP error fetching plugin from https://get.pulumi.com/releases/plugins/pulumi-resource-aws-v2.13.1-darwin-arm64.tar.gz

cli version: v2.23.1

roderik · 2021-03-22T13:12:10Z

I have been running it since launch on M1 without any issues.

Some things I would check (with examples from my setup)

package.json:

  "devDependencies": {
    "@types/lodash.merge": "4.6.6",
    "@types/node": "14.14.35",
    "prettier": "2.2.1",
    "typescript": "4.2.3"
  },
  "dependencies": {
    "@pulumi/aws": "3.34.1",
    "@pulumi/awsx": "0.25.0",
    "@pulumi/azure": "3.51.0",
    "@pulumi/cloudflare": "2.14.1",
    "@pulumi/eks": "0.22.0",
    "@pulumi/gcp": "4.15.0",
    "@pulumi/kubernetes": "2.8.3",
    "@pulumi/pulumi": "2.23.1",
    "lodash.merge": "4.6.2",
    "utility-types": "3.10.0"
  }

So you are getting an old version installed, while you define a new version. I would first recommend not using the ^ in your dependencies as you have more control over what version you actually get. But that is beside the point now.

Then if you use Yarn run yarn why @pulumi/aws:

yarn why v1.22.10
[1/4] 🤔  Why do we have the module "@pulumi/aws"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "@pulumi/aws@3.34.1"
info Has been hoisted to "@pulumi/aws"
info Reasons this module exists
   - Specified in "dependencies"
   - Hoisted from "@pulumi#eks#@pulumi#aws"
info Disk size without dependencies: "26.53MB"
info Disk size with unique dependencies: "139.66MB"
info Disk size with transitive dependencies: "222.11MB"
info Number of shared dependencies: 28
✨  Done in 0.29s.

I think you will either find here, 2 versions, or just the old version. It will tell you the reason it selected a version.

If you use npm i do not know an equivalent command, but you can open the package-lock.json and find all references to pulumu/aws

    "node_modules/@pulumi/aws": {
      "version": "v3.34.1",
      "hasInstallScript": true,
      "license": "Apache-2.0",
      "dependencies": {
        "@pulumi/pulumi": "^2.17.0",
        "aws-sdk": "^2.0.0",
        "builtin-modules": "3.0.0",
        "mime": "^2.0.0",
        "read-package-tree": "^5.2.1",
        "resolve": "^1.7.1"
      }
    },
    "node_modules/@pulumi/awsx": {
      "version": "v0.25.0+dirty",
      "license": "Apache-2.0",
      "dependencies": {
        "@pulumi/docker": "^1.0.0 || ^2.0.0",
        "@types/aws-lambda": "^8.10.23",
        "mime": "^2.0.0"
      },
      "peerDependencies": {
        "@pulumi/aws": "^3.25.1",
        "@pulumi/pulumi": "^1.9.1 || ^2.0.0"
      }
    },

Look for instances where something depends on pulumi/aws and check the version there. In essence that is the same info that yarn why returns. The lock files define the versions installed.

When you found one, you can probably find a node_modules folder in that package folder in the main node_modules. You then have to figure out why it depends on that old version. It might be because that one does not have a caret in its version and forces an older version.

Now in the pulumi stack (get the json by using pulumi stack export > tmp.json) you have something like

            {
                "urn": "urn:pulumi:production::bpaas-clusters::pulumi:providers:aws::default_3_33_0",
                "custom": true,
                "id": "659acefd-e182-47a9-bdd7-e26d64b87e96",
                "type": "pulumi:providers:aws",
                "inputs": {
                    "region": "eu-west-3",
                    "version": "3.33.0"
                },
                "outputs": {
                    "region": "eu-west-3",
                    "version": "3.33.0"
                }
            },

In your case default_2_13_1, that, if you have the right versions npm installed, will auto-upgrade, but is not. You could try to modify it to 3_33_0 and 3.33.0 everywhere it mentions default_2_13_1 and import it again (pulumi stack import --file tmp.json) but wait for Pulumi confirmation that this does not break everything.

Then again, if it cannot find the 3.33.0 version now, forcing it to use it will not really help ofcourse.

Another thing you could investigate is your path.

which pulumi -> /Users/roderik/.pulumi/bin/pulumi
which node -> /Users/roderik/.nvm/versions/node/v15.11.0/bin/node
echo $PATH -> /usr/local/Caskroom/google-cloud-sdk/latest/google-cloud-sdk/bin:/Users/roderik/.nvm/versions/node/v15.11.0/bin:/opt/homebrew/Cellar/zplug/2.4.2/bin:/Users/roderik/bin:/opt/homebrew/bin:/usr/local/bin:/opt/homebrew/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin:/opt/homebrew/opt/zplug/repos/zpm-zsh/colorize/bin:/Users/roderik/.pulumi/bin

While typing this, just thought of a different thing, this is the flow to upgrade

update package.json
yarn/npm install
run pulumi up.

From your error messages i think you might have forgotten to do 2.

Adam-Burke · 2021-04-06T05:01:22Z

Yeah everything seems to be fine from a dependecy standpoint. We use pnpm but there is a version of why and it shows the correct version. Similarly it's correct both in node_modules and pnpm-lock.yaml.

But yeah the pulumi export file still references the older versions. I updated them in our staging env and it seems to the newer installed version and that seems to have unblocked things. Seems a bit risky, but I'll live with that.

Thanks for your help.

runlevel5 · 2021-08-27T03:51:46Z

It would be uber awesome if docker images also offer amr64 variants

Ref: https://github.com/pulumi/pulumi/issues/7740

jaxxstorm self-assigned this Jun 22, 2020

lukehoban mentioned this issue Sep 28, 2020

[Feature request] ARM64-based CLI package #5466

Closed

lukehoban changed the title ~~Create an ARM based build for the pulumi cli~~ Create an ARM based build for the pulumi cli (and plugins) Nov 10, 2020

lukehoban mentioned this issue Dec 4, 2020

Strange error messages when performing pulumi up or pulumi destroy #5859

Closed

lukehoban mentioned this issue Jan 3, 2021

[ARM] unsupported plugin architecture: arm64 pulumi/pulumi-aws#1283

Closed

stack72 assigned stack72 and unassigned jaxxstorm Jan 19, 2021

stack72 added this to the 0.51 milestone Jan 19, 2021

stack72 mentioned this issue Jan 19, 2021

try an arm64 build #5729

Merged

stack72 removed this from the 0.51 milestone Jan 21, 2021

leezen modified the milestones: 0.51, 0.52 Jan 26, 2021

leezen removed this from the 0.52 milestone Feb 2, 2021

kherings mentioned this issue Feb 5, 2021

runtime stalls when using pulumi with Rosetta2 emulation on Apple M1 #6261

Closed

lukehoban mentioned this issue Mar 6, 2021

arm64 unsupported pulumi/pulumi-auth0#66

Closed

stack72 added a commit that referenced this issue Mar 10, 2021

[cli] Add ability to consume arm64 plugins from pulumi cli

9e6314c

Related: #4868 Also adds the arm64 build and deployment steps via goreleaser

stack72 mentioned this issue Mar 10, 2021

[cli] Add ability to consume arm64 plugins from pulumi cli #6492

Merged

stack72 added a commit that referenced this issue Mar 10, 2021

[cli] Add ability to consume arm64 plugins from pulumi cli

d16e4fc

Related: #4868 Also adds the arm64 build and deployment steps via goreleaser

stack72 added a commit that referenced this issue Mar 10, 2021

[cli] Add ability to consume arm64 plugins from pulumi cli (#6492)

dc7eb5e

Related: #4868 Also adds the arm64 build and deployment steps via goreleaser

stack72 added kind/enhancement Improvements or new features resolution/fixed This issue was fixed labels Mar 12, 2021

stack72 closed this as completed Mar 12, 2021

leezen added this to the 0.53 milestone Mar 12, 2021

robertvanhoesel mentioned this issue Nov 8, 2021

Unable to download plugin random for darwin-arm64 pre v3.1 #8378

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create an ARM based build for the pulumi cli (and plugins) #4868

Create an ARM based build for the pulumi cli (and plugins) #4868

jaxxstorm commented Jun 22, 2020 •

edited by stack72

justinvp commented Jun 22, 2020

lukehoban commented Jun 22, 2020

lukehoban commented Jun 22, 2020

jaxxstorm commented Sep 11, 2020

lukehoban commented Sep 28, 2020

justinvp commented Nov 10, 2020

jaxxstorm commented Nov 11, 2020

lukehoban commented Nov 11, 2020 •

edited

roderik commented Dec 28, 2020

kherings commented Jan 8, 2021 •

edited

stack72 commented Jan 21, 2021

roderik commented Jan 22, 2021

jamesla commented Mar 4, 2021 •

edited

stack72 commented Mar 4, 2021

stack72 commented Mar 10, 2021

roderik commented Mar 10, 2021

roderik commented Mar 10, 2021 •

edited

stack72 commented Mar 12, 2021

stack72 commented Mar 17, 2021

barakcoh commented Mar 18, 2021 •

edited

stack72 commented Mar 18, 2021

Adam-Burke commented Mar 22, 2021 •

edited

roderik commented Mar 22, 2021

Adam-Burke commented Apr 6, 2021

runlevel5 commented Aug 27, 2021

Create an ARM based build for the pulumi cli (and plugins) #4868

Create an ARM based build for the pulumi cli (and plugins) #4868

Comments

jaxxstorm commented Jun 22, 2020 • edited by stack72

justinvp commented Jun 22, 2020

lukehoban commented Jun 22, 2020

lukehoban commented Jun 22, 2020

jaxxstorm commented Sep 11, 2020

lukehoban commented Sep 28, 2020

justinvp commented Nov 10, 2020

jaxxstorm commented Nov 11, 2020

lukehoban commented Nov 11, 2020 • edited

roderik commented Dec 28, 2020

kherings commented Jan 8, 2021 • edited

stack72 commented Jan 21, 2021

roderik commented Jan 22, 2021

jamesla commented Mar 4, 2021 • edited

stack72 commented Mar 4, 2021

stack72 commented Mar 10, 2021

roderik commented Mar 10, 2021

roderik commented Mar 10, 2021 • edited

stack72 commented Mar 12, 2021

stack72 commented Mar 17, 2021

barakcoh commented Mar 18, 2021 • edited

stack72 commented Mar 18, 2021

Adam-Burke commented Mar 22, 2021 • edited

roderik commented Mar 22, 2021

Adam-Burke commented Apr 6, 2021

runlevel5 commented Aug 27, 2021

jaxxstorm commented Jun 22, 2020 •

edited by stack72

lukehoban commented Nov 11, 2020 •

edited

kherings commented Jan 8, 2021 •

edited

jamesla commented Mar 4, 2021 •

edited

roderik commented Mar 10, 2021 •

edited

barakcoh commented Mar 18, 2021 •

edited

Adam-Burke commented Mar 22, 2021 •

edited