Documentation for state file versioning considerations

[edahlseng]

I have a general question about how Terraform handles state file versions. I couldn't find this information anywhere in the online docs; I hope I haven't just missed it! Please point me in the right direction if I should be asking this somewhere else!

I notice that if I create a remote state file in one version of Terraform (e.g. on my local computer, using v0.11.8), and then try to run a plan/apply against that state file in a lower version of Terraform (e.g. as part of a CI build, using v0.11.1), Terraform complains with an error message along the lines of "this state file was created with a newer version of Terraform." This suggests to me that state files have an explicit version stored with them. I don't understand the full implications of this, however. For example, are we only limited with using older versions of Terraform? Can we always use a newer version of Terraform to operate on a given state file? Is there a process for upgrading the version of the state file? Would there be any benefit in doing such an operation, or is that a silly idea to even consider?

[apparentlymart]

Hi @edahlseng!

I think you're right that there isn't a single place where all of the details here are documented right now. For now, I'll try to summarize the situation here and then I'll label this as a documentation issue to remind us to write something more explicit about it the docs in future.

A key piece of context here is that the state contains a mixture of data generated by Terraform Core and data generated by provider plugins. Since provider plugins are on a separate release cycle than Terraform Core, there are two different version handling mechanisms here to deal with those two problems, though both have similar high-level behavior.

Terraform Core controls the overall structure of the state file, the syntax of which is versioned explicitly using its own version number. The current state file format as I write this is version 3, and Terraform v0.12 will introduce version 4. Terraform Core generally knows how to upgrade forwards through state format versions, but an older version of Terraform cannot downgrade a state to read it.

The version number I covered in the previous paragraph represents the syntax of the state file format. The file can also include some semantic details that emerged from the behavior of Terraform in a particular version, and so as a safety mechanism to avoid unpredictable behavior we additionally require the use of a version of Terraform at least as new as the one that most recently updated the state. As Terraform's behavior stabilizes in future versions it is likely that this constraint will be relaxed and we will use only the file format version number, but the current implementation is conservative to reduce the risk of problems.

Alongside both of those mechanisms, each separate resource instance in the state has data that is structured in a way decided by the provider, and so this structure has its own per-resource-type version number that the provider manages. If a breaking change is made to the state format for a particular resource type then the next provider release will increase this version number and include some logic to upgrade from prior versions. Just as with the core format version, upgrades can only move forwards and so once you have run terraform apply with a newer version of a provider it may not be possible to work with resources it created or updated in an older provider version.

terraform plan allows you to see what the effect of a change will be without taking those actions or updating the state. This means you can try out new versions of Terraform and providers without getting locked in to those new versions immediately. Once you've run terraform apply it may no longer be possible to downgrade.

[edahlseng]

Thanks for the helpful answer @apparentlymart! I'm starting to understand how Terraform handles the state file much better now.

To make sure I'm fully grokking everything, are the following statements correct?

• In a Terraform state file, there are three "types of versions":
  • The syntax version of the state file itself
  • The versions of the provider used to create each of the resources
  • The Terraform version used to initially create the state file
• The versions of a state file may change due to the following (listed in the same order as above):
  • An updated version of the Terraform core works with the state file
  • An updated provider is used to apply resources
  • This one can't change, as the state file is only every created for the first time once
• The versions within a state file can only be updated (and are only updated) by running a terraform apply

[apparentlymart]

That is very close! Two refinements:

The version on Terraform stored is the most recent one that applied changes, not the one that started it.

Also, any command that modifies the state may upgrade one of these versions. terraform apply is the main situation, but terraform refresh is another way the state of resources can be updated, with the same effect. The terraform state family of subcommands work entirely in Terraform Core so they can update the format version and the Core version but will not change the provider (resource schema) versions in there.

[edahlseng]

Thanks for the clarifications @apparentlymart!

[laurapacilio]

@apparentlymart Just wanted to check in on this, since it's been a few years. I found a bit about state here: https://www.terraform.io/docs/language/state/index.html but wasn't sure if we wanted to circle back to this to update the docs further, or if this has been resolved and we can close. Thank you!

[anthonylavado]

This seems to be, so far, the only definitive thing I've found about the version fields in the state file. I would say it's worth a mention in the documentation, because it's not entirely clear there (and a language reference should be thorough wherever possible).

[laurapacilio]

noted! Thank you both. I'm going to make a ticket for myself for this work. Behind the scenes, @apparentlymart and I were discussing a possible revamp of the entire state section. I'm not sure I can fit that in this quarter, but I'm going to add it to the documentation backlog so we make a note of it and can plan it in the next couple of months. :-)

[apparentlymart]

The main thing that changed since my original writeup above is that we did subsequently reach the point I was alluding to where Terraform's behavior is fixed enough that we can rely only on versioning of the state snapshot syntax and no longer check that a snapshots was created by the current or an earlier version of Terraform CLI.

The current situation is therefore a little simpler:

• Terraform Core uses the top-level version number to know how to interpret the rest of the JSON data structure. Terraform v1.0 uses state snapshot format version 4 and we have no current plans to introduce any new versions, though of course that may change over time as we investigate solutions to new requirements.

  (State snapshot version 3 is, in effect, no longer supported by Terraform v0.13 and later, because it has no way to represent the hierarchical provider source addresses we now use to support third-party provider installation. The v0.13 upgrade guide includes a step of running terraform apply with Terraform v0.13, which means you can't pass this point without your latest state snapshot using format version 4. For all intents and purposes then, at the time I'm writing this snapshot version 4 is the only supported format.)

• As part of the process of planning changes to the remote objects represented by resource instances in the state, Terraform Core will send the raw data produced by the most recent action on each instance to the provider, along with the resource-instance-specific schema version number. The provider's job then is to return an equivalent object that matches the provider's current schema, or to return an error if the existing data has an unsupported schema version.

  Terraform Core doesn't do anything with the resource-instance-specific schema version except to send it to the provider in this upgrade request. The treatment of that number therefore depends on the provider itself. By convention providers typically support upgrading older schema versions to the current schema version, but will return an error if the stored schema version is newer than current, which suggests that the object was created by a newer version of the provider.

I still don't have a good sense of how best to incorporate this into the docs, so again I'm leaving this here in the hope that it's useful for those who are interested. @laurapacilio, let's talk about this again some more when you have some time to dig in!