Way to move resources from one state to another.

[tmccombs]

Terraform Version

any

Use Cases

Consider the problem of splitting up a monolithic terraform configuration into smaller root modules. This requires either splitting up the existing state, or tearing down and rebuilding all resources. In many cases the latter isn't possible. See for example https://medium.com/@calinflorescu3/splitting-a-monolithic-terraform-state-d26e866de629

Another possible use case is if you are refactoring something, and want to move a resource from one root module to a different one.

Attempted Solutions

There are a couple of ways you can split up the state currently, but they aren't great. And in particular involve doing things that terraform documentation explicitly recommends against.

One is you can manually edit the state files, or write scripts to modify the state files. But there is relatively little documentation of the format of the state files, and the documentation states "direct file editing of the state is discouraged."

Another option is to do a state pull on both states, then do a state mv using the --state and --state-out options, followed by using state push. This was the method used in the blog post above. But again the terraform documantion says the --state and --state-out options are "legacy features ... that we no longer recommend". And it is unclear that these options will be supported in future versions.

Finally, it is possible to do a state rm on the source state and an import on the destination state for each resource. However, not all resources support import. And there isn't currently a way to do bulk imports, which can make the import portion pretty slow.

Proposal

One option is not to change any functionality, but "bless" one of the first two above methods for moving resources between different states. Meaning, that rather than having a blanket recommendation against modifying state directly or using the legacy options, the documentation states cases where it is ok to use them (like moving resources between state).

Alternatively either change the existing state mv command or add a new command that can move a resource from one state to another, even for non-local backends. I'm not sure exactly how you would specify the options for the backends. Perhaps give a reference to another file (or directory?) containing the configuration for the destination backend? Ideally this option would also support doing a bulk operation so that many resources/modules could be moved at once.

References

No response

[smarek]

@tmccombs you seen this? https://technotrampoline.com/articles/renaming-and-moving-things-in-terraform/

[tmccombs]

That is just renaming resources within the same state file/workspace.

I'm talking about the case where I have a single monolithic workspace/root configuration module/state that I want to split up into multiple smaller ones.

[tmccombs]

I suppose another workaround for creating new root modules is to copy the entire state, and rm every resource I don't want in the new configuration.

[apparentlymart]

Thanks for sharing this use-case, @tmccombs.

A significant design problem that has got in the way of designing solutions for this so far is that Terraform CLI is fundamentally designed around the idea that each working directory has only one active backend and one active workspace and therefore only one active state at a time. We lack the primitives required to talk about something moving from one workspace to another within a single backend or between workspaces in different backends.

Right now I think the most compelling solution is to pull all of the different state snapshots locally and to manually edit them in a text editor, which is of course not ideal.

It would in principle be possible to develop an external tool that can shuffle around objects inside state files while leaving most of the data unmodified. I think the main challenges would be the same challenges that arise with trying to do this with the existing state surgery commands: moving things around can mess up the relationships between objects, such as the resource dependency information and the information about which provider configuration each resource belongs to, and so the result is not guaranteed to be equivalent to the input.

The workaround of copying the entire state snapshot and removing parts of it you don't want anymore while leaving all of the individual resource instance addresses unchanged is probably the most robust approach we have available right now, because that will at least preserve the subset of dependency-related information that concerns the objects that aren't being removed. After completing that pruning you can then use moved to reshuffle the addresses of resource instances to final locations, with Terraform rewriting the dependencies based on the configuration in that case.

---

A complete and thorough solution to this problem is unfortunately rather challenging even putting aside the Terraform CLI design problems, because moving objects between different state snapshots in isolation is a fundamentally "incomplete" operation: an object in a state snapshot tracks its dependencies on other objects in the same state snapshot, and so moving objects around invalidates those relationships.

The moved feature avoids this problem within a single configuration because it can take into account both the configuration and the state at the same time, and so it can see from the configuration how all of the objects are supposed to be connected. A cross-configuration version of that would require access to both configurations in addition to access to the latest state snapshots of each.

A problem with having a "blessed" solution is that it must typically then be a complete and reliable one: folks who devise their own workarounds typically accept that they are workarounds and thus their behavior may not be complete but are "complete enough" to get things done, but once something is a "real feature" that is documented people reasonably expect it to to be complete, correct, and thorough, which means that we need to at least have a design for how to make it so, even if the first iteration of it only implements part of that design. So far it isn't clear how something like this ought to work, but we'll keep thinking about this as Terraform evolves and look for opportunities to improve it.

[omarismail]

Hey @tmccombs ,

My name is Omar and I'm the PM for Terraform. I'm exploring this problem space and would love to chat more. Mind emailing me and we can setup a time to chat if you are interested? oismail@hashicorp.com

[tareksamni]

You could also use terraform-state-split cli which helps a lot with the process in big scale projects.

https://github.com/shebang-labs/terraform-state-split

Demo:
https://asciinema.org/a/qqF2E5Uz2ybwzhJdMpuufzblu

Install:

brew tap shebang-labs/tap
brew install terraform-state-split

[cristian-vanti]

I have exactly the same problem. Moreover, terraform is invoked by our Gitlab pipelines that do not includes steps to run custom terraform mv or terraform import steps. I am uncomfortable at using manual tasks in each environment, including production. Within all the options that I considered, deleting the resources from a state and creating them again into other states could be the least painful. But still quite painful

[cristian-vanti]

@tmccombs did you find any acceptable options?

[marco-m-pix4d]

@tmccombs , @omarismail
have a look at

• https://github.com/minamijoyo/tfmigrate
• https://github.com/pix4D/terravalet (by yours truly)

[edmundcraske-cgi]

I did manage to make use of import and removed blocks which are available since Terraform 1.7.5 (iirc) to be able to move resources between two different state files - worked fairly well!

[bartpalmowski-jt]

I did manage to make use of import and removed blocks which are available since Terraform 1.7.5 (iirc) to be able to move resources between two different state files - worked fairly well!

That's not moving but rather importing and deleting (unless I'm missing something).

[edmundcraske-cgi]

I did manage to make use of import and removed blocks which are available since Terraform 1.7.5 (iirc) to be able to move resources between two different state files - worked fairly well!

That's not moving but rather importing and deleting (unless I'm missing something).

I do feel like you are perhaps missing something - as from the point of view of the old state, you were tracking a resource which you no longer wish to track, so you wish to remove it; and from the point of view of the new state, you are not tracking a resource that you wish to track, so you wish to import it. Terraform does not have an innate way of managing more than one state at a time - a given state is an inherent part of the context in which Terraform runs, so the desire to 'move' a resource from one state to another is not something that can be done atomically within a single Terraform run, as a single Terraform run only operates on one state.