Feature request: taking "ownership" of existing resources

[gliptak]

I would like to propose a "take ownership" modifier added to lifecycle flags.

When a resource cannot be created as it already exists (e.g. same name, a 409 return status), the resource/state still would be recorded in tfstate and updated as configured in tf files.

This augments (in the "front") the current lifecycle keys:

https://www.terraform.io/docs/configuration/resources.html#create_before_destroy

• create_before_destroy
• prevent_destroy
• ignore_changes

allowing for handling autogenerated AWS resources, but this is applicable to resources in general.

This also could be used to (gradually) move already (manually or using a different configuration tool) generated resources under the Terraform tfstate.

Terraform Version

terraform -version
Terraform v0.8.5

[apparentlymart]

Hi @gliptak!

This is intended to be solved using the import system, though it doesn't yet have 100% support across all resources.

The idea is to write a resource config block for the thing you want to import, and then run the terraform import command to tell Terraform which pre-existing object the config block is intended to manage. Then on future runs of terraform plan the diff will be made against that existing object, rather than wanting to create a new one.

Does that get you what you need here, or is there something else I'm missing?

[gliptak]

The import system doesn't cover the initial usecase, as these resources are created dynamically during running an apply. So they do not exist to import beforehand, and the apply fails (as they are duplicates at that time, or cannot be present in the configuration). Adding this flag to lifecycle does seem to improve user experience (augmenting or maybe even replacing import functionality).

[apparentlymart]

I'm not sure I understand the subtle behavior you're thinking of, @gliptak.

You say they don't exist beforehand, but yet they are duplicates during apply. Are you talking then about a situation where two resources in the same configuration conflict with each other? If we allowed Terraform to bind the same object to two different resource instances then that would presumably result in non-convergent behavior, because the two resources would compete with each other to update the same object.

Is there a specific resource type you have in mind here? Perhaps a more concrete example would help me to understand what you're looking to do here.

[gliptak]

@apparentlymart Thank you for reviewing this with me. I'm not expecting see multiple names for the same resource, but hoping to be able to create and/or take ownership of a resource.

The concrete example I ran into was creating an AWS Lambda, which created an associated Log Group in the "background", on which I wanted to set the Expiry Days for. When adding the Log Group to the tf configuration, I observed 409 conflict returned by AWS (as Terraform is expecting to be able to create the resource, as contrasted to creating and/or taking ownership of).

[apparentlymart]

Oh, I understand now what you mean: you're talking about ancillary resources that get created as a side-effect of creating a main resource, which Terraform then can't see.

Today the only way to make this work is to first apply to create the primary resource, and then subsequently import the others. But I understand that this is not sufficient for your use-case, since it forces you to always do work in multiple steps rather than creating the whole stack from scratch in one apply.

In your specific case, I wonder if it would work to have Terraform create the log group first, exploiting the fact that they have a predictable naming scheme built from the lambda function name, and then create the Lambda function. You could achieve that with some clever use of depends_on. Perhaps that can get you what you need in the short term, though I agree that what you've described here is a wart in Terraform's design that it would be nice to solve.

[gliptak]

Yes, reordering sounds like a good workaround. Thank you for the suggestion.

[mitchellh]

This is an interesting use case but I worry it is a bit unusual. I think in situations like this it isn't too much to ask for a multi-run convergence. If you have an idea for syntax/style to make something like this work I'd be open to it. I think one way we can make this work is perhaps somehow allowing import as a resource type within Terraform itself, though it is odd. It should be possible to play around with something like that.

Due to the unusual ask here I'm going to close this. If there is more demand and someone wants to try to take a look at this, they're welcome to!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.