Cannot import a Resource provided by a Dynamic Resource Provider

[garrettklatte]

What happened?

It is not possible to import a Resource that is provided by a Dynamic Resource Provider written with the Pulumi Python SDK due to the following error: Exception calling application: '__provider'

Example

To reproduce, use the following versions:

$ python --version
Python 3.12.3

$ pulumi version
v3.115.2

$ pip freeze
Arpeggio==2.0.2
attrs==23.2.0
dill==0.3.8
grpcio==1.60.1
parver==0.5
protobuf==4.25.3
pulumi==3.115.2
PyYAML==6.0.1
semver==2.13.0
six==1.16.0

The python-resource-type-name example provider can demonstrate this issue. Copy the Pulumi.yaml and __main__.py files to your local workspace.

Run pulumi up to generate the stack.

Then, attempt to import a resource. The name and ID of the resource are inconsequential.

$ pulumi import pulumi-python:dynamic/custom-provider:CustomResource CustomResourceImport bc50618b-d053-49a4-8704-40d72b645a2a
Previewing import (import-testing-resource):
     Type                                                     Name                                                   Plan       Info
     pulumi:pulumi:Stack                                      dynamic_resource_type_name_py-import-testing-resource             1 error
 =   └─ pulumi-python:dynamic/custom-provider:CustomResource  CustomResourceImport                                   import     1 error

Diagnostics:
  pulumi:pulumi:Stack (dynamic_resource_type_name_py-import-testing-resource):
    error: preview failed

  pulumi-python:dynamic/custom-provider:CustomResource (CustomResourceImport):
    error: Preview failed: Exception calling application: '__provider'

You can reproduce this issue by importing from a file as well. Create a file with the following contents:

{
  "resources": [
    {
      "id": "bc50618b-d053-49a4-8704-40d72b645a2a",
      "name": "CustomResourceImport",
      "type": "pulumi-python:dynamic/custom-provider:CustomResource"
    }
  ]
}

You will get the same error:

$ pulumi import -f import-no-provider.json
Previewing import (import-testing-resource):
     Type                                                     Name                                                   Plan       Info
     pulumi:pulumi:Stack                                      dynamic_resource_type_name_py-import-testing-resource             1 error
 =   └─ pulumi-python:dynamic/custom-provider:CustomResource  CustomResourceImport                                   import     1 error

Diagnostics:
  pulumi:pulumi:Stack (dynamic_resource_type_name_py-import-testing-resource):
    error: preview failed

  pulumi-python:dynamic/custom-provider:CustomResource (CustomResourceImport):
    error: Preview failed: Exception calling application: '__provider'

It does not matter if you specify an existing provider. Create a file with the following contents (note that the URN of your provider depends on your stack name):

{
  "nameTable": {
    "custom-provider": "urn:pulumi:import-testing-resource::dynamic_resource_type_name_py::pulumi:providers:pulumi-python::default"
  },
  "resources": [
    {
      "id": "bc50618b-d053-49a4-8704-40d72b645a2a",
      "name": "CustomResourceImport",
      "provider": "custom-provider",
      "type": "pulumi-python:dynamic/custom-provider:CustomResource"
    }
  ]
}

You will get the same error:

$ pulumi import -f import.json
Previewing import (import-testing-resource):
     Type                                                     Name                                                   Plan       Info
     pulumi:pulumi:Stack                                      dynamic_resource_type_name_py-import-testing-resource             1 error
 =   └─ pulumi-python:dynamic/custom-provider:CustomResource  CustomResourceImport                                   import     1 error

Diagnostics:
  pulumi-python:dynamic/custom-provider:CustomResource (CustomResourceImport):
    error: Preview failed: Exception calling application: '__provider'

  pulumi:pulumi:Stack (dynamic_resource_type_name_py-import-testing-resource):
    error: preview failed

The issue also appears if you try to import a resource within your source code directly. Add the following lines to the bottom of your __main__.py file (and import ResourceOptions atop the file):

imported_resource = CustomResource("imported-resource", opts=ResourceOptions(import_="doesn't matter")
export("imported-urn", imported_resource.urn)

You will get the same error:

$ pulumi up
Previewing update (import-testing-resource):
     Type                                                     Name                                                   Plan       Info
     pulumi:pulumi:Stack                                      dynamic_resource_type_name_py-import-testing-resource             1 error
 =   └─ pulumi-python:dynamic/custom-provider:CustomResource  imported-resource                                      import     1 error

Diagnostics:
  pulumi-python:dynamic/custom-provider:CustomResource (imported-resource):
    error: Preview failed: Exception calling application: '__provider'

  pulumi:pulumi:Stack (dynamic_resource_type_name_py-import-testing-resource):
    error: preview failed

Output of pulumi about

CLI
Version 3.115.2
Go Version go1.22.2
Go Compiler gc

Plugins
KIND NAME VERSION
language python unknown

Host
OS darwin
Version 14.4.1
Arch arm64

This project is written in python: executable='/Users/gklatte/pulumi-import-testing/venv/bin/python3' version='3.12.3'

Current Stack: organization/dynamic_resource_type_name_py/import-testing-resource

TYPE URN
pulumi:pulumi:Stack urn:pulumi:import-testing-resource::dynamic_resource_type_name_py::pulumi:pulumi:Stack::dynamic_resource_type_name_py-import-testing-resource
pulumi:providers:pulumi-python urn:pulumi:import-testing-resource::dynamic_resource_type_name_py::pulumi:providers:pulumi-python::default
pulumi-python:dynamic/custom-provider:CustomResource urn:pulumi:import-testing-resource::dynamic_resource_type_name_py::pulumi-python:dynamic/custom-provider:CustomResource::resource-name

Found no pending operations associated with import-testing-resource

Backend
Name MLLAXP1SXQ05N
URL s3://redacted?region=us-east-1&profile=redacted
User gklatte
Organizations
Token type personal

Dependencies:
NAME VERSION
parver 0.5
pip 24.0
pulumi 3.115.2

Pulumi locates its logs in /var/folders/5m/409zz62n0vdgbxb7nhcp6zl80000gp/T/ by default

Additional context

(Caveat: I lack familiarity with the internals of Pulumi but here is what I have observed so far)

The Read method of DynamicResourceProviderServicer receives a request without any properties. As a result, an exception is raised when trying to access the provider. The request received by DynamicResourceProviderServicer should contain a __provider property but does not.

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[Frassle]

Yeh this is a limitation with dynamic providers. We might be able to fix the last scenario (using the Import resource option) but as far as I know the others aren't possible to fix.

[garrettklatte]

Yeh this is a limitation with dynamic providers. We might be able to fix the last scenario (using the Import resource option) but as far as I know the others aren't possible to fix.

So it is not possible for a dynamic provider, using any of the Pulumi SDKs, to import resources?

[jallison-relaytx]

Yeh this is a limitation with dynamic providers. We might be able to fix the last scenario (using the Import resource option) but as far as I know the others aren't possible to fix.

Having that last option, using ResourceOptions, would be useful for my use case if it is indeed doable.

My path for getting here was:

• Have resources in a cloud provider manually created and managed
• See it become untenable to manage manually and decided to make life better by writing some dynamic resource providers
• Attempt to import said resources

I was a little surprised this issue is only 3 weeks old as I would imagine this to be a somewhat common path. In my case, deleting/recreating resources is possible but it would be a large QOL improvement if I could import resources since there is historical data associated with them.

[garrettklatte]

I was a little surprised this issue is only 3 weeks old as I would imagine this to be a somewhat common path

I too was surprised to not find an existing issue documenting this problem. My work around has been to write a Pulumi Native provider for my resources instead of using a Dynamic Resource provider. That work is ongoing but I am hopeful that the path is smoother.

[IdoOzeri]

+1, I'm having the same issue here. Is there any workaround?

EDIT: actually, I was able to work around it by first exporting the stack's state by pulumi stack export > state.json, editing it to include the imported resources I needed, and then pulumi stack import --file state.json.

This resulted in my resources imported to the dynamic provider. Not pretty, error-prone - but gets the job done.

[justinvp]

Note that Node.js dynamic providers have the same limitation.