Import resources into Terraform

[piavlo]

Use case , manage infrastructure environments both existing and created from scratch, by same terraform configs. For example in development environments we want to create everything from scratch and destroy everything then we finish working with it. In production we want to be able to add new resources to it with terraform but not conflict with resources that already exist but not managed by terrafrom yet.

For ex:

resource "aws_subnet" "b" {
    vpc_id = "${aws_vpc.terraform.id}"
    cidr_block = "${var.vpc_network_prefix}.${lookup(var.vpc_subnet_suffix, "b")}"
    map_public_ip_on_launch = true
    availability_zone = "${var.region}b"
    if defined(var.subnet_b_id) { load_by_id(${var.subnet_b_id}) }

}

if terraform has not yet created this resource it will check if subnet_b_id is defined and if resource with such id exists in provider api it will import this resource into terraform based on data in api instead of creating it.

[sparkprime]

I think this doesn't need language support, instead you just need the ability to import the existing subnet into the tfstate file (and give it the name "b" in the process). Then Terraform would reuse the existing one instead of creating a new one to fill the "b" slot. I.e. that conditional if defined(...) essentially already exists in the core terraform planning logic to check if a resource already exists in the tfstate.

[nogweii]

Would the recommended way forward for this be then to create a script that traverses an AWS network, getting all Route53 records, VPC information, instance data, etc, and generating the appropriate tfstate JSON + example .tf files?

[armon]

Correct. Ideally the tool would be able to scan resources on various provides and with operator assistance map it to config logical names.

[carimura]

+1 to this one for us. Keeps us from being able to launch new clusters of our service into an existing VPC. Any update on this one since Dec?

[phinze]

No specific update on the feature itself, it's still on the roadmap.

But I wanted to jump in here to link an excellent article by @phrawzty on a strategy for dealing with existing resources with current versions of terraform:

https://www.dark.ca/2015/01/27/handling-extant-resources-in-terraform/

[clstokes]

+1, would love to see this (soon).

[AlexanderEkdahl]

An alternative approach could be something inspired by terraform taint where the user could execute something along the lines of terraform add [options] name id. For example terraform add aws_security_group.name sg-4bdc302e and it would be marked as existing but without attributes in the .tfstate-file. Upon the next refresh or apply the attributes would be populated.

[johnrengelman]

I like @AlexanderEkdahl suggestion. This would be awesome and we could keep people out of the state file - @mitchellh @catsby, thoughts? I'd be willing to do a POC for something like this.

[mitchellh]

Thats a pretty neat idea. Could help basic stuff... What do you think @phinze?

[phinze]

Agreed - I think that would be a really valuable baby step to take. 👣 POC away @johnrengelman!

[johnrengelman]

Cool.

[johnrengelman]

I put together a very simple POC for discussion - #2022

[blalor]

Separate from @johnrengelman's POC (where I commented, too), I feel like a missing part of this conversation is actually generating the Terraform config. I'd expect it to still be a relatively manual, iterative process, but I'd want terraform import <resource_id> <provider_id> to generate config appropriate for a .tf file, probably with a helper for the various _association resource types. Even outputting JSON a la .tfjson would be a huge help.

[ketzacoatl]

+1 to @blalor's comment, though I would advocate for taking this in separate.. the first being coverage for inport to begin with.. generating config has more nuances and work in general.

[meylor]

Others may find this tool helpful. I tested the EC2, ELB, security group, and VPC capabilities and it was able to extract existing AWS infrastructure into resources. https://github.com/dtan4/terraforming

[jwaldrip]

👍 I would love to see this incorporated into terraform.

[nrcxcia]

Would love to see this as well, this functionality would allow us to start using terraform at my company to manage existing infrastructure. +1

[ckelner]

👍

[danielcbright]

👍

[smcavoy-travelbird]

I'll join the party. This would be a huge feature.
(AWS first please)

[JigarS91]

I am trying to create new subnets in existing vpc.
I have added vpc id in variables.tf file as

variable "aws_vpc" {
default = "vpc id"
}

Getting below error:-
aws_subnet.private: Error creating subnet: InvalidVpcID.NotFound: The vpc ID "vpc id not found"
Not sure how to go ahead with this one. Need help !

[preflightsiren]

@JigarS91 https://www.terraform.io/community.html is the place to ask these questions :)

[rizvfe00]

👍

[avdhoot]

+1 it will be good to have feature in terraform.

[cbroglie]

terraforming is working OK for me too. My process is essentially to export all resources into tfstate form, then rename the ones I'm importing, and delete the others. Would be great to have this functionality built into terraform.

[leighmhart]

This is a bit of a show stopper for my application - I don't want to edit the tfstate if I can avoid it... any update on whether this will be moving forward in one form or another?

[Pryz]

👍

[robertwe]

👍

[TerribleDev]

👍

[LusciousPear]

+1, this makes it much easier to recover from partial applies

[cyakimov]

👍

[davidfic]

👍

[jvasallo]

👍

[nitdana]

+1

[manojlds]

Folks, github has had reactions for a while. Please stop the +1 and thumbs up.

[brandon-dacrib]

seconded on a request for an update, clearly this is something that the community wants. If the response is that hey this will be included in version 0.6.20 then so be it.

[johnrengelman]

Looking at the work on the master branch, parts of this (or maybe all) will be in 0.7.

[mitchellh]

The basics of this are in master and will be part of 0.7. See the website subfolder for docs until it is released. More fancy features such as config generation will come, but 0.7 will be able to import resources into Terraform.

This is just the beginning, but we're finally getting there!

[ozbillwang]

Anyone interesting the converter, you need to follow up this ticket:

hashicorp/hcl#162

[sergeylanzman]

You can try new solution https://github.com/GoogleCloudPlatform/terraformer
Full import existing resources, include tf and tfstate. Full support cloud providers options
Support Google cloud and aws

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.