New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import resources into Terraform #581

Closed
piavlo opened this Issue Nov 19, 2014 · 57 comments

Comments

Projects
None yet
@piavlo

piavlo commented Nov 19, 2014

Use case , manage infrastructure environments both existing and created from scratch, by same terraform configs. For example in development environments we want to create everything from scratch and destroy everything then we finish working with it. In production we want to be able to add new resources to it with terraform but not conflict with resources that already exist but not managed by terrafrom yet.

For ex:

resource "aws_subnet" "b" {
    vpc_id = "${aws_vpc.terraform.id}"
    cidr_block = "${var.vpc_network_prefix}.${lookup(var.vpc_subnet_suffix, "b")}"
    map_public_ip_on_launch = true
    availability_zone = "${var.region}b"
    if defined(var.subnet_b_id) { load_by_id(${var.subnet_b_id}) }

}

if terraform has not yet created this resource it will check if subnet_b_id is defined and if resource with such id exists in provider api it will import this resource into terraform based on data in api instead of creating it.

@sethvargo sethvargo changed the title from import resources into terraform to Import resources into Terraform Nov 19, 2014

@sparkprime

This comment has been minimized.

Show comment
Hide comment
@sparkprime

sparkprime Nov 27, 2014

Contributor

I think this doesn't need language support, instead you just need the ability to import the existing subnet into the tfstate file (and give it the name "b" in the process). Then Terraform would reuse the existing one instead of creating a new one to fill the "b" slot. I.e. that conditional if defined(...) essentially already exists in the core terraform planning logic to check if a resource already exists in the tfstate.

Contributor

sparkprime commented Nov 27, 2014

I think this doesn't need language support, instead you just need the ability to import the existing subnet into the tfstate file (and give it the name "b" in the process). Then Terraform would reuse the existing one instead of creating a new one to fill the "b" slot. I.e. that conditional if defined(...) essentially already exists in the core terraform planning logic to check if a resource already exists in the tfstate.

@evaryont

This comment has been minimized.

Show comment
Hide comment
@evaryont

evaryont Dec 9, 2014

Would the recommended way forward for this be then to create a script that traverses an AWS network, getting all Route53 records, VPC information, instance data, etc, and generating the appropriate tfstate JSON + example .tf files?

evaryont commented Dec 9, 2014

Would the recommended way forward for this be then to create a script that traverses an AWS network, getting all Route53 records, VPC information, instance data, etc, and generating the appropriate tfstate JSON + example .tf files?

@armon

This comment has been minimized.

Show comment
Hide comment
@armon

armon Dec 9, 2014

Member

Correct. Ideally the tool would be able to scan resources on various provides and with operator assistance map it to config logical names.

Member

armon commented Dec 9, 2014

Correct. Ideally the tool would be able to scan resources on various provides and with operator assistance map it to config logical names.

@carimura

This comment has been minimized.

Show comment
Hide comment
@carimura

carimura Feb 23, 2015

+1 to this one for us. Keeps us from being able to launch new clusters of our service into an existing VPC. Any update on this one since Dec?

carimura commented Feb 23, 2015

+1 to this one for us. Keeps us from being able to launch new clusters of our service into an existing VPC. Any update on this one since Dec?

@phinze

This comment has been minimized.

Show comment
Hide comment
@phinze

phinze Mar 2, 2015

Member

No specific update on the feature itself, it's still on the roadmap.

But I wanted to jump in here to link an excellent article by @phrawzty on a strategy for dealing with existing resources with current versions of terraform:

https://www.dark.ca/2015/01/27/handling-extant-resources-in-terraform/

Member

phinze commented Mar 2, 2015

No specific update on the feature itself, it's still on the roadmap.

But I wanted to jump in here to link an excellent article by @phrawzty on a strategy for dealing with existing resources with current versions of terraform:

https://www.dark.ca/2015/01/27/handling-extant-resources-in-terraform/

@clstokes

This comment has been minimized.

Show comment
Hide comment
@clstokes

clstokes Apr 27, 2015

Contributor

+1, would love to see this (soon).

Contributor

clstokes commented Apr 27, 2015

+1, would love to see this (soon).

@AlexanderEkdahl

This comment has been minimized.

Show comment
Hide comment
@AlexanderEkdahl

AlexanderEkdahl Apr 27, 2015

Contributor

An alternative approach could be something inspired by terraform taint where the user could execute something along the lines of terraform add [options] name id. For example terraform add aws_security_group.name sg-4bdc302e and it would be marked as existing but without attributes in the .tfstate-file. Upon the next refresh or apply the attributes would be populated.

Contributor

AlexanderEkdahl commented Apr 27, 2015

An alternative approach could be something inspired by terraform taint where the user could execute something along the lines of terraform add [options] name id. For example terraform add aws_security_group.name sg-4bdc302e and it would be marked as existing but without attributes in the .tfstate-file. Upon the next refresh or apply the attributes would be populated.

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman May 12, 2015

Contributor

I like @AlexanderEkdahl suggestion. This would be awesome and we could keep people out of the state file - @mitchellh @catsby, thoughts? I'd be willing to do a POC for something like this.

Contributor

johnrengelman commented May 12, 2015

I like @AlexanderEkdahl suggestion. This would be awesome and we could keep people out of the state file - @mitchellh @catsby, thoughts? I'd be willing to do a POC for something like this.

@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh May 12, 2015

Member

Thats a pretty neat idea. Could help basic stuff... What do you think @phinze?

Member

mitchellh commented May 12, 2015

Thats a pretty neat idea. Could help basic stuff... What do you think @phinze?

@phinze

This comment has been minimized.

Show comment
Hide comment
@phinze

phinze May 12, 2015

Member

Agreed - I think that would be a really valuable baby step to take. 👣 POC away @johnrengelman!

Member

phinze commented May 12, 2015

Agreed - I think that would be a really valuable baby step to take. 👣 POC away @johnrengelman!

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman May 12, 2015

Contributor

Cool.

Contributor

johnrengelman commented May 12, 2015

Cool.

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman May 20, 2015

Contributor

I put together a very simple POC for discussion - #2022

Contributor

johnrengelman commented May 20, 2015

I put together a very simple POC for discussion - #2022

@blalor

This comment has been minimized.

Show comment
Hide comment
@blalor

blalor May 21, 2015

Contributor

Separate from @johnrengelman's POC (where I commented, too), I feel like a missing part of this conversation is actually generating the Terraform config. I'd expect it to still be a relatively manual, iterative process, but I'd want terraform import <resource_id> <provider_id> to generate config appropriate for a .tf file, probably with a helper for the various _association resource types. Even outputting JSON a la .tfjson would be a huge help.

Contributor

blalor commented May 21, 2015

Separate from @johnrengelman's POC (where I commented, too), I feel like a missing part of this conversation is actually generating the Terraform config. I'd expect it to still be a relatively manual, iterative process, but I'd want terraform import <resource_id> <provider_id> to generate config appropriate for a .tf file, probably with a helper for the various _association resource types. Even outputting JSON a la .tfjson would be a huge help.

@ketzacoatl

This comment has been minimized.

Show comment
Hide comment
@ketzacoatl

ketzacoatl May 31, 2015

Contributor

+1 to @blalor's comment, though I would advocate for taking this in separate.. the first being coverage for inport to begin with.. generating config has more nuances and work in general.

Contributor

ketzacoatl commented May 31, 2015

+1 to @blalor's comment, though I would advocate for taking this in separate.. the first being coverage for inport to begin with.. generating config has more nuances and work in general.

@meylor

This comment has been minimized.

Show comment
Hide comment
@meylor

meylor Jun 5, 2015

Others may find this tool helpful. I tested the EC2, ELB, security group, and VPC capabilities and it was able to extract existing AWS infrastructure into resources. https://github.com/dtan4/terraforming

meylor commented Jun 5, 2015

Others may find this tool helpful. I tested the EC2, ELB, security group, and VPC capabilities and it was able to extract existing AWS infrastructure into resources. https://github.com/dtan4/terraforming

@jwaldrip

This comment has been minimized.

Show comment
Hide comment
@jwaldrip

jwaldrip Jul 8, 2015

Contributor

👍 I would love to see this incorporated into terraform.

Contributor

jwaldrip commented Jul 8, 2015

👍 I would love to see this incorporated into terraform.

@nrcxcia

This comment has been minimized.

Show comment
Hide comment
@nrcxcia

nrcxcia Jul 9, 2015

Would love to see this as well, this functionality would allow us to start using terraform at my company to manage existing infrastructure. +1

nrcxcia commented Jul 9, 2015

Would love to see this as well, this functionality would allow us to start using terraform at my company to manage existing infrastructure. +1

@ckelner

This comment has been minimized.

Show comment
Hide comment
@ckelner

ckelner commented Jul 20, 2015

👍

@danielcbright

This comment has been minimized.

Show comment
Hide comment
@danielcbright

danielcbright commented Jul 20, 2015

👍

@smcavoy-travelbird

This comment has been minimized.

Show comment
Hide comment
@smcavoy-travelbird

smcavoy-travelbird Jul 22, 2015

I'll join the party. This would be a huge feature.
(AWS first please)

smcavoy-travelbird commented Jul 22, 2015

I'll join the party. This would be a huge feature.
(AWS first please)

@aldarund

This comment has been minimized.

Show comment
Hide comment
@aldarund

aldarund commented Jan 11, 2016

+1

@rokka-n

This comment has been minimized.

Show comment
Hide comment
@rokka-n

rokka-n commented Jan 26, 2016

👍

@JigarS91

This comment has been minimized.

Show comment
Hide comment
@JigarS91

JigarS91 Feb 1, 2016

I am trying to create new subnets in existing vpc.
I have added vpc id in variables.tf file as

variable "aws_vpc" {
default = "vpc id"
}

Getting below error:-
aws_subnet.private: Error creating subnet: InvalidVpcID.NotFound: The vpc ID "vpc id not found"
Not sure how to go ahead with this one. Need help !

JigarS91 commented Feb 1, 2016

I am trying to create new subnets in existing vpc.
I have added vpc id in variables.tf file as

variable "aws_vpc" {
default = "vpc id"
}

Getting below error:-
aws_subnet.private: Error creating subnet: InvalidVpcID.NotFound: The vpc ID "vpc id not found"
Not sure how to go ahead with this one. Need help !

@preflightsiren

This comment has been minimized.

Show comment
Hide comment
@preflightsiren

preflightsiren commented Feb 4, 2016

@JigarS91 https://www.terraform.io/community.html is the place to ask these questions :)

@rizvfe00

This comment has been minimized.

Show comment
Hide comment
@rizvfe00

rizvfe00 commented Feb 18, 2016

👍

@avdhoot

This comment has been minimized.

Show comment
Hide comment
@avdhoot

avdhoot Feb 29, 2016

+1 it will be good to have feature in terraform.

avdhoot commented Feb 29, 2016

+1 it will be good to have feature in terraform.

@cbroglie

This comment has been minimized.

Show comment
Hide comment
@cbroglie

cbroglie Mar 14, 2016

Contributor

terraforming is working OK for me too. My process is essentially to export all resources into tfstate form, then rename the ones I'm importing, and delete the others. Would be great to have this functionality built into terraform.

Contributor

cbroglie commented Mar 14, 2016

terraforming is working OK for me too. My process is essentially to export all resources into tfstate form, then rename the ones I'm importing, and delete the others. Would be great to have this functionality built into terraform.

@allandrick

This comment has been minimized.

Show comment
Hide comment
@allandrick

allandrick Mar 24, 2016

This is a bit of a show stopper for my application - I don't want to edit the tfstate if I can avoid it... any update on whether this will be moving forward in one form or another?

allandrick commented Mar 24, 2016

This is a bit of a show stopper for my application - I don't want to edit the tfstate if I can avoid it... any update on whether this will be moving forward in one form or another?

@Pryz

This comment has been minimized.

Show comment
Hide comment
@Pryz

Pryz Mar 30, 2016

Contributor

👍

Contributor

Pryz commented Mar 30, 2016

👍

@robertwe

This comment has been minimized.

Show comment
Hide comment
@robertwe

robertwe commented Apr 1, 2016

👍

@TerribleDev

This comment has been minimized.

Show comment
Hide comment
@TerribleDev

TerribleDev commented Apr 15, 2016

👍

@LusciousPear

This comment has been minimized.

Show comment
Hide comment
@LusciousPear

LusciousPear Apr 20, 2016

+1, this makes it much easier to recover from partial applies

LusciousPear commented Apr 20, 2016

+1, this makes it much easier to recover from partial applies

@cyakimov

This comment has been minimized.

Show comment
Hide comment
@cyakimov

cyakimov commented May 3, 2016

👍

@davidfic

This comment has been minimized.

Show comment
Hide comment
@davidfic

davidfic commented May 3, 2016

👍

@jvasallo

This comment has been minimized.

Show comment
Hide comment
@jvasallo

jvasallo May 12, 2016

Contributor

👍

Contributor

jvasallo commented May 12, 2016

👍

@nitdana

This comment has been minimized.

Show comment
Hide comment
@nitdana

nitdana commented May 17, 2016

+1

@manojlds

This comment has been minimized.

Show comment
Hide comment
@manojlds

manojlds May 17, 2016

Contributor

Folks, github has had reactions for a while. Please stop the +1 and thumbs up.

Contributor

manojlds commented May 17, 2016

Folks, github has had reactions for a while. Please stop the +1 and thumbs up.

@brandon-dacrib

This comment has been minimized.

Show comment
Hide comment
@brandon-dacrib

brandon-dacrib Jun 10, 2016

seconded on a request for an update, clearly this is something that the community wants. If the response is that hey this will be included in version 0.6.20 then so be it.

brandon-dacrib commented Jun 10, 2016

seconded on a request for an update, clearly this is something that the community wants. If the response is that hey this will be included in version 0.6.20 then so be it.

@johnrengelman

This comment has been minimized.

Show comment
Hide comment
@johnrengelman

johnrengelman Jun 10, 2016

Contributor

Looking at the work on the master branch, parts of this (or maybe all) will be in 0.7.

Contributor

johnrengelman commented Jun 10, 2016

Looking at the work on the master branch, parts of this (or maybe all) will be in 0.7.

@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh Jun 10, 2016

Member

The basics of this are in master and will be part of 0.7. See the website subfolder for docs until it is released. More fancy features such as config generation will come, but 0.7 will be able to import resources into Terraform.

This is just the beginning, but we're finally getting there!

Member

mitchellh commented Jun 10, 2016

The basics of this are in master and will be part of 0.7. See the website subfolder for docs until it is released. More fancy features such as config generation will come, but 0.7 will be able to import resources into Terraform.

This is just the beginning, but we're finally getting there!

@ozbillwang

This comment has been minimized.

Show comment
Hide comment
@ozbillwang

ozbillwang Jun 23, 2017

Anyone interesting the converter, you need to follow up this ticket:

hashicorp/hcl#162

ozbillwang commented Jun 23, 2017

Anyone interesting the converter, you need to follow up this ticket:

hashicorp/hcl#162

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment