How do I get the creation time of a terraform run/state file?

[red8888]

Terraform v0.11.7

Sorry if this isn't the right place for this but I guess i would consider this an issue because I cant find anything in the docs for this.

In cloudformation I can just do a "describe-stacks" and get the creation time, how do I do this with terraform? This is super simple and easy and I can use it to script destroying CF stacks older than a period of time.

For example if I have my state setup like this:

terraform {
  backend "s3" {
    bucket = "mybucket"
    region = "us-east-1"
    key = "myapps/app01/branch_name"
  }
}

I want to have a job that runs every 10 mins that gets the creation time of those "myapps/app01/branch_name" state files and destroys the ones that are older than 1 month.

How can I accomplish this like I can with CF?

[red8888]

I'm leaning toward writing a script to do the following hacky thing to get that CF behavior (which would essentially be just two commands):

1. List S3 state files "myapps/app01/branch_*"
2. Compare last modified date of S3 keys to current time and create a list of "expired" files
3. For each file dynamically write a TF file with backend config for that key because running with command line args seems to be broken
   This command "succeeds" but it doesn't init any tf modules, create a .terraform folder, nor pull down a state file or anything:
   terraform init -backend-config='region=us-east-1' -backend-config='bucket=mybucket' -backend-config='myapps/app01/branch_name'
4. For each file run terraform init, then terraform destroy, delete .terraform/terraform.tfstate, and delete the tf file (because we're generating a new one each time)

Surely there is a better way?

I suppose if I could point to a state file directly all with command line args and just destroy (without having to generate a tf file) that would make things much easier. but doing terraform init like that doesnt work for me

EDIT: ok I forgot I ALWAYS need a tf file when running init and the official way is to have one with just 'terraform { backend "s3" {} }' in it. Which is... kind of insane haha. At least that part isn't as bad though

[apparentlymart]

Hi @red8888,

Terraform doesn't currently retain metadata about creation and update times itself. Users don't usually want to automatically delete Terraform-managed infrastructure, but I can see that there are some more unsual use-cases where that would be desirable; it sounds like you're using Terraform to manage some development or other temporary infrastructure here?

Since Terraform itself doesn't understand the relationships between your state files (Terraform just reads/writes that individual object from S3, ignoring everything else in the bucket, including the metadata), I think indeed something like what you proposed here is the best option. S3 itself knows when those resources are created, and your external program can "know" how you structure the objects in that bucket to know what is and is not a Terraform state.

This is one way in which Terraform's design differs from CloudFormation's: Terraform doesn't have a first-class concept of a "stack", so each user defines their own organizational structure for storing the different states, depending on their organizational needs. An implication of that difference is that Terraform itself can't list "stacks", and so unfortunately a tool that operates "for each stack" is outside of Terraform's scope.

Destroying all of the resources in state does require access to the configuration today, because certain objects (provider configurations in particular, but also destroy-time provisioners, etc) are kept only there and not in the state. The primary use-case for Terraform is applying changes to configuration files managed in version control, and Terraform is less suited for situations where configuration is being dynamically generated on the fly and where, in particular, it's not possible to reproduce the configuration that created a particular set of resources.

It feels to me like CloudFormation may be a better fit for your use-case, since it stores the whole configuration and state within the stack object and so makes it easier to do this "for each stack, destroy it conditionally" sort of operation on temporary/transient infrastructure.

[red8888]

Thank you for the thoughtful reply i appreciate you explaining in detail. This is not an issue then as much as a guidance question.

I think my problem is that terraform is a provisioning tool but I'm trying to use it as a deployment tool-am I framing this correctly?

Terraform still has "plan" and is more predictable then CF for many operations. What I'm leaning toward now is building the ECS clusters, ECR repos, task definitions, etc with TF because that will be static per each app per environment. Then for actually deploying the containers/ECS services using CF because that will be per app per environment per branch and dynamic and I can list those stacks and even auto expire them.

Or is what I'm doing just insane? I haven't seen examples of others using TF and CF in conjunction like this. Any tips/suggests would be helpful!

[apparentlymart]

I think the distinction you made here is the right one, and what we usually recommend. The way we'd usually put this is, approximately:

• Use Terraform to provision your infrastructure, which is generally long-lived and changes only rarely in response to direct changes to configuration.
• Include some sort of "scheduler" (ECS, Kubernetes, Nomad, ...) as part of your infrastructure.
• Deploy applications (or more generically, "workloads") via the scheduler.

When using Kubernetes and Nomad users will often just talk directly to their own APIs for deployment. I don't have any hands-on experience running ECS in production, so I'm not sure if that direct management would be appropriate there or if having CloudFormation manage that deployment is the more common approach, but in your case it sounds like the extra layer of a CloudFormation stack might be useful as a way to manage the expiration behavior you want to implement.

Some teams do use Terraform to kick off a deployment in a scheduler, e.g. by using the nomad_job resource to ask Nomad to start running a set of tasks. In this case that part would usually be in a separate Terraform configuration whose lifecycle is independent from the one that provisioned the infrastructure, and Terraform is just acting as a more convenient interface for calling into the scheduler's API. In your case, with the dynamic expiration behavior, I think using CloudFormation for this part of the problem seems plausible since it plays to CloudFormation's strengths while letting you still use Terraform for the longer-lived infrastructure.

[hashibot]

Hello again!

We didn't hear back from you, so I'm going to close this in the hope that a previous response gave you the information you needed. If not, please do feel free to re-open this and leave another comment with the information my human friends requested above. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.