Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pulumi import by passing json file is failing #2596

Closed
yogeshghotekar opened this issue Oct 5, 2023 · 7 comments · Fixed by #2605
Closed

Pulumi import by passing json file is failing #2596

yogeshghotekar opened this issue Oct 5, 2023 · 7 comments · Fixed by #2605
Assignees
Labels
impact/regression Something that used to work, but is now broken kind/bug Some behavior is incorrect or out of spec p1 A bug severe enough to be the next item assigned to an engineer resolution/fixed This issue was fixed
Milestone

Comments

@yogeshghotekar
Copy link

yogeshghotekar commented Oct 5, 2023

What happened?

I am trying to import resource define in json file to pulumi but is failing to import the helm charts
This started failing on pulumi-kubernetes provider v4.3.0 It works fine with v4.2.0

Example

To reproduce this issue. You can create resources in json file. In my case I am creating this resource json file from my terraform deployed resources and then trying to import it in pulumi.

json file

{
    "nameTable": {
        "edge": "urn:pulumi:production::deployer::myinfra:deployer:Cluster::edge",
        "edge-provider": "urn:pulumi:production::deployer::pulumi:providers:kubernetes::edge"
    },
    "resources": [
       {
            "name": "edge/metallb-system",
            "type": "kubernetes:core/v1:Namespace",
            "id": "metallb-system",
            "parent": "edge",
            "provider": "edge-provider"
        },
		{
            "name": "edge/metallb-system/my-secret",
            "type": "kubernetes:core/v1:Secret",
            "id": "metallb-system/my-secret",
            "parent": "edge",
            "provider": "edge-provider"
        },
        {
            "name": "edge/metallb-system/metallb",
            "type": "kubernetes:helm.sh/v3:Release",
            "id": "metallb-system/metallb",
            "parent": "edge",
            "provider": "edge-provider"
        }
    ]
}

my resource creation code

release, err := helmv3.NewRelease(ctx, fmt.Sprintf("%s/%s/%s", clusterName, helmRelease.Namespace, helmRelease.Name),
                &helmv3.ReleaseArgs{
                    Name:            pulumi.String("metallb"),
                    Chart:           pulumi.String("metallb-system/metallb"),
                    Namespace:       pulumi.StringPtr("metallb-system"),
                    CreateNamespace: pulumi.BoolPtr(true),
                    WaitForJobs:     pulumi.BoolPtr(true),
                    SkipAwait:       pulumi.BoolPtr(false),
                    CleanupOnFail:   pulumi.BoolPtr(true),
                    Atomic:          pulumi.BoolPtr(false),
                    Version:         pulumi.StringPtr("1.18.0"),
                    Timeout:         pulumi.IntPtr(900),
                    Values:          pulumi.ToMap(helmRelease.Values),
                    Lint:            pulumi.BoolPtr(false),
                    Verify:          pulumi.BoolPtr(false),
                    ResetValues:     pulumi.BoolPtr(true),
                    AllowNullValues: pulumi.BoolPtr(true),
                },
                pulumi.Provider(providers[clusterName]))

once json file is ready execute below command
pulumi import -f ./vars/bulk-import.json --protect=false --generate-code=false

Error:

kubernetes:helm.sh/v3:Release (edge/metallb-system/metallb):
error: Preview failed: failed to load chart from temp directory: stat metallb: no such file or directory

This works fine with pulumi-kuberenetes provider v4.2.0 and failing on v4.3.0

Output of pulumi about

CLI
Version 3.85.0
Go Version go1.21.1
Go Compiler gc

Plugins
NAME VERSION
go unknown

Host
OS debian
Version 12.1
Arch x86_64

This project is written in go: executable='/usr/local/go/bin/go' version='go version go1.20.8 linux/amd64'

Additional context

Suspecting the issue is due to changes #2568 added in v4.3.0 I tried ignoreChanges: ["checksum"] but still it is failing for pulumi import

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@yogeshghotekar yogeshghotekar added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Oct 5, 2023
@mikhailshilkov
Copy link
Member

@EronWright This sounds like a regression from #2568, could you take a look?

@EronWright
Copy link
Contributor

EronWright commented Oct 10, 2023

I am able to reproduce this issue, and I agree that it is a regression from #2568.

Some repro notes (WIP):

❯ k create namespace metallb-system
namespace/metallb-system created
❯ helm repo add metallb https://metallb.github.io/metallb
❯ helm install metallb metallb/metallb -n metallb-system

@mikhailshilkov mikhailshilkov added p1 A bug severe enough to be the next item assigned to an engineer impact/regression Something that used to work, but is now broken and removed needs-triage Needs attention from the triage team labels Oct 10, 2023
@mikhailshilkov mikhailshilkov added this to the 0.95 milestone Oct 10, 2023
@EronWright

This comment was marked as resolved.

@EronWright
Copy link
Contributor

To provide a brief update, a potential fix has been implemented and is being tested:
#2605

EronWright added a commit that referenced this issue Oct 20, 2023
<!--Thanks for your contribution. See [CONTRIBUTING](CONTRIBUTING.md)
    for Pulumi's contribution guidelines.

    Help us merge your changes more quickly by adding more details such
    as labels, milestones, and reviewers.-->


### Proposed changes

This is a series of commits to fix a p1 and to improve the import of
Helm Releases. The regression occurred because `Read` attempted to use
the provider-generated inputs to compute a checksum, and returned an
error if it couldn't, and this caused the `pulumi import` tool to fail
in cases where the inputs aren't guessable. The fix is to eat the
problem during `Read`.

While implementing the fix, it became apparent that the templating code
for the `Release` resource should be decoupled from the `Chart`
resource, mainly to improve the consistency of how chart names are
handled.

Specific changes:
- suppress error in computing inputs during `Read`.
- decouple the templating code.
- simplify the chart linting code.
- use a timeout that is consistent with the Delete op.
- new tests for the import variants.

<!--Give us a brief description of what you've done and what it solves.
-->
Closes #2596

### Related issues (optional)

<!--Refer to related PRs or issues: #1234, or 'Fixes #1234' or 'Closes
#1234'.
Or link to full URLs to issues or pull requests in other GitHub
repositories. -->

---------

Co-authored-by: Ramon Quitales <ramon@pulumi.com>
@pulumi-bot pulumi-bot added the resolution/fixed This issue was fixed label Oct 20, 2023
@EronWright
Copy link
Contributor

@yogeshghotekar we published a fix in v4.5.0, could you reply with a confirmation that the fix was effective for you?

@rhuanbarreto
Copy link

This is still actual in v4.5.4. We should reopen

@EronWright
Copy link
Contributor

@rhuanbarreto could you elaborate on the specific issue that you are seeing? Some output would really help, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact/regression Something that used to work, but is now broken kind/bug Some behavior is incorrect or out of spec p1 A bug severe enough to be the next item assigned to an engineer resolution/fixed This issue was fixed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants