Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import resources into Terraform #581

Closed
piavlo opened this issue Nov 19, 2014 · 59 comments
Closed

Import resources into Terraform #581

piavlo opened this issue Nov 19, 2014 · 59 comments
Labels

Comments

@piavlo
Copy link

@piavlo piavlo commented Nov 19, 2014

Use case , manage infrastructure environments both existing and created from scratch, by same terraform configs. For example in development environments we want to create everything from scratch and destroy everything then we finish working with it. In production we want to be able to add new resources to it with terraform but not conflict with resources that already exist but not managed by terrafrom yet.

For ex:

resource "aws_subnet" "b" {
    vpc_id = "${aws_vpc.terraform.id}"
    cidr_block = "${var.vpc_network_prefix}.${lookup(var.vpc_subnet_suffix, "b")}"
    map_public_ip_on_launch = true
    availability_zone = "${var.region}b"
    if defined(var.subnet_b_id) { load_by_id(${var.subnet_b_id}) }

}

if terraform has not yet created this resource it will check if subnet_b_id is defined and if resource with such id exists in provider api it will import this resource into terraform based on data in api instead of creating it.

@sethvargo sethvargo changed the title import resources into terraform Import resources into Terraform Nov 19, 2014
@sparkprime
Copy link
Contributor

@sparkprime sparkprime commented Nov 27, 2014

I think this doesn't need language support, instead you just need the ability to import the existing subnet into the tfstate file (and give it the name "b" in the process). Then Terraform would reuse the existing one instead of creating a new one to fill the "b" slot. I.e. that conditional if defined(...) essentially already exists in the core terraform planning logic to check if a resource already exists in the tfstate.

@nogweii
Copy link

@nogweii nogweii commented Dec 9, 2014

Would the recommended way forward for this be then to create a script that traverses an AWS network, getting all Route53 records, VPC information, instance data, etc, and generating the appropriate tfstate JSON + example .tf files?

@armon
Copy link
Member

@armon armon commented Dec 9, 2014

Correct. Ideally the tool would be able to scan resources on various provides and with operator assistance map it to config logical names.

@carimura
Copy link

@carimura carimura commented Feb 23, 2015

+1 to this one for us. Keeps us from being able to launch new clusters of our service into an existing VPC. Any update on this one since Dec?

@phinze
Copy link
Member

@phinze phinze commented Mar 2, 2015

No specific update on the feature itself, it's still on the roadmap.

But I wanted to jump in here to link an excellent article by @phrawzty on a strategy for dealing with existing resources with current versions of terraform:

https://www.dark.ca/2015/01/27/handling-extant-resources-in-terraform/

@clstokes
Copy link
Contributor

@clstokes clstokes commented Apr 27, 2015

+1, would love to see this (soon).

@AlexanderEkdahl
Copy link
Contributor

@AlexanderEkdahl AlexanderEkdahl commented Apr 27, 2015

An alternative approach could be something inspired by terraform taint where the user could execute something along the lines of terraform add [options] name id. For example terraform add aws_security_group.name sg-4bdc302e and it would be marked as existing but without attributes in the .tfstate-file. Upon the next refresh or apply the attributes would be populated.

@johnrengelman
Copy link
Contributor

@johnrengelman johnrengelman commented May 12, 2015

I like @AlexanderEkdahl suggestion. This would be awesome and we could keep people out of the state file - @mitchellh @catsby, thoughts? I'd be willing to do a POC for something like this.

@mitchellh
Copy link
Member

@mitchellh mitchellh commented May 12, 2015

Thats a pretty neat idea. Could help basic stuff... What do you think @phinze?

@phinze
Copy link
Member

@phinze phinze commented May 12, 2015

Agreed - I think that would be a really valuable baby step to take. 👣 POC away @johnrengelman!

@johnrengelman
Copy link
Contributor

@johnrengelman johnrengelman commented May 12, 2015

Cool.

@johnrengelman
Copy link
Contributor

@johnrengelman johnrengelman commented May 20, 2015

I put together a very simple POC for discussion - #2022

@blalor
Copy link
Contributor

@blalor blalor commented May 21, 2015

Separate from @johnrengelman's POC (where I commented, too), I feel like a missing part of this conversation is actually generating the Terraform config. I'd expect it to still be a relatively manual, iterative process, but I'd want terraform import <resource_id> <provider_id> to generate config appropriate for a .tf file, probably with a helper for the various _association resource types. Even outputting JSON a la .tfjson would be a huge help.

@ketzacoatl
Copy link
Contributor

@ketzacoatl ketzacoatl commented May 31, 2015

+1 to @blalor's comment, though I would advocate for taking this in separate.. the first being coverage for inport to begin with.. generating config has more nuances and work in general.

@meylor
Copy link

@meylor meylor commented Jun 5, 2015

Others may find this tool helpful. I tested the EC2, ELB, security group, and VPC capabilities and it was able to extract existing AWS infrastructure into resources. https://github.com/dtan4/terraforming

@jwaldrip
Copy link
Contributor

@jwaldrip jwaldrip commented Jul 8, 2015

👍 I would love to see this incorporated into terraform.

@nrcxcia
Copy link

@nrcxcia nrcxcia commented Jul 9, 2015

Would love to see this as well, this functionality would allow us to start using terraform at my company to manage existing infrastructure. +1

@ckelner
Copy link

@ckelner ckelner commented Jul 20, 2015

👍

1 similar comment
@danielcbright
Copy link

@danielcbright danielcbright commented Jul 20, 2015

👍

@smcavoy-travelbird
Copy link

@smcavoy-travelbird smcavoy-travelbird commented Jul 22, 2015

I'll join the party. This would be a huge feature.
(AWS first please)

@JigarS91
Copy link

@JigarS91 JigarS91 commented Feb 1, 2016

I am trying to create new subnets in existing vpc.
I have added vpc id in variables.tf file as

variable "aws_vpc" {
default = "vpc id"
}

Getting below error:-
aws_subnet.private: Error creating subnet: InvalidVpcID.NotFound: The vpc ID "vpc id not found"
Not sure how to go ahead with this one. Need help !

@preflightsiren
Copy link

@preflightsiren preflightsiren commented Feb 4, 2016

@JigarS91 https://www.terraform.io/community.html is the place to ask these questions :)

@rizvfe00
Copy link

@rizvfe00 rizvfe00 commented Feb 18, 2016

👍

@avdhoot
Copy link

@avdhoot avdhoot commented Feb 29, 2016

+1 it will be good to have feature in terraform.

@cbroglie
Copy link
Contributor

@cbroglie cbroglie commented Mar 14, 2016

terraforming is working OK for me too. My process is essentially to export all resources into tfstate form, then rename the ones I'm importing, and delete the others. Would be great to have this functionality built into terraform.

@allandrick
Copy link

@allandrick allandrick commented Mar 24, 2016

This is a bit of a show stopper for my application - I don't want to edit the tfstate if I can avoid it... any update on whether this will be moving forward in one form or another?

@Pryz
Copy link
Contributor

@Pryz Pryz commented Mar 30, 2016

👍

1 similar comment
@robertwe
Copy link

@robertwe robertwe commented Apr 1, 2016

👍

@TerribleDev
Copy link

@TerribleDev TerribleDev commented Apr 15, 2016

👍

@LusciousPear
Copy link

@LusciousPear LusciousPear commented Apr 20, 2016

+1, this makes it much easier to recover from partial applies

@cyakimov
Copy link

@cyakimov cyakimov commented May 3, 2016

👍

3 similar comments
@davidfic
Copy link

@davidfic davidfic commented May 3, 2016

👍

@jvasallo
Copy link
Contributor

@jvasallo jvasallo commented May 12, 2016

👍

@nitdana
Copy link

@nitdana nitdana commented May 17, 2016

+1

@manojlds
Copy link
Contributor

@manojlds manojlds commented May 17, 2016

Folks, github has had reactions for a while. Please stop the +1 and thumbs up.

@brandon-dacrib
Copy link

@brandon-dacrib brandon-dacrib commented Jun 10, 2016

seconded on a request for an update, clearly this is something that the community wants. If the response is that hey this will be included in version 0.6.20 then so be it.

@johnrengelman
Copy link
Contributor

@johnrengelman johnrengelman commented Jun 10, 2016

Looking at the work on the master branch, parts of this (or maybe all) will be in 0.7.

@mitchellh
Copy link
Member

@mitchellh mitchellh commented Jun 10, 2016

The basics of this are in master and will be part of 0.7. See the website subfolder for docs until it is released. More fancy features such as config generation will come, but 0.7 will be able to import resources into Terraform.

This is just the beginning, but we're finally getting there!

@ozbillwang
Copy link

@ozbillwang ozbillwang commented Jun 23, 2017

Anyone interesting the converter, you need to follow up this ticket:

hashicorp/hcl#162

@sergeylanzman
Copy link

@sergeylanzman sergeylanzman commented May 3, 2019

You can try new solution https://github.com/GoogleCloudPlatform/terraformer
Full import existing resources, include tf and tfstate. Full support cloud providers options
Support Google cloud and aws

@hashibot
Copy link

@hashibot hashibot bot commented Jul 26, 2019

I'm going to lock this issue because it has been closed for 30 days . This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@hashibot hashibot bot locked and limited conversation to collaborators Jul 26, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

You can’t perform that action at this time.