Error: with canonical "aws_security_group.this_name_prefix": graph node already exists #103

PePoDev · 2021-02-20T17:33:27Z

Terraform version : 0.14.7
OS: Ubuntu 20 on WSL
Provider: AWS
Command: terraform state pull | inframap generate --tfstate --connections=false | dot -Tpng > inframap.png

First I got empty graph, Then I try with --connections=false and got error

Error: with canonical "aws_security_group.this_name_prefix": graph node already exists

I think you need this output terraform state list | grep aws_security_group.this_name_prefix

module.alb_sg.aws_security_group.this_name_prefix[0]
module.allow_memcached_sg.aws_security_group.this_name_prefix[0]
module.grafana_sg.aws_security_group.this_name_prefix[0]
module.graylog_sg.aws_security_group.this_name_prefix[0]
module.moodle_sg.aws_security_group.this_name_prefix[0]
module.rds_mariadb_sg.aws_security_group.this_name_prefix[0]

The text was updated successfully, but these errors were encountered:

rpgd60 · 2021-02-22T06:34:49Z

I found a similar error when trying to import a tfstate file.

$ ./inframap version
The current version is: v0.5.2-1-g86f936f

$ ./inframap generate --tfstate ./terraform.tfstate 
Error: with canonical "aws_security_group_rule.egress_rules": graph node already exists
Usage:
(...)

To reproduce:

OS : Kubuntu 20.10
Provider AWS
Terraform version : v0.14.7
The .tfstate file was generated using the Terraform "locals" tutorial at
https://learn.hashicorp.com/tutorials/terraform/locals?in=terraform/configuration-language
with some minor modifications

$ terraform version
Terraform v0.14.7
+ provider registry.terraform.io/hashicorp/aws v3.29.0
+ provider registry.terraform.io/hashicorp/random v3.1.0

The list of resources in the tfstate file:

$ terraform state list
data.aws_ami.amazon_linux
data.aws_availability_zones.available
aws_instance.app[0]
aws_instance.app[1]
random_string.lb_id
module.vpc.aws_eip.nat[0]
module.vpc.aws_eip.nat[1]
module.vpc.aws_internet_gateway.this[0]
module.vpc.aws_nat_gateway.this[0]
module.vpc.aws_nat_gateway.this[1]
module.vpc.aws_route.private_nat_gateway[0]
module.vpc.aws_route.private_nat_gateway[1]
module.vpc.aws_route.public_internet_gateway[0]
module.vpc.aws_route_table.private[0]
module.vpc.aws_route_table.private[1]
module.vpc.aws_route_table.public[0]
module.vpc.aws_route_table_association.private[0]
module.vpc.aws_route_table_association.private[1]
module.vpc.aws_route_table_association.public[0]
module.vpc.aws_route_table_association.public[1]
module.vpc.aws_subnet.private[0]
module.vpc.aws_subnet.private[1]
module.vpc.aws_subnet.public[0]
module.vpc.aws_subnet.public[1]
module.vpc.aws_vpc.this[0]
module.app_security_group.module.sg.aws_security_group.this_name_prefix[0]
module.app_security_group.module.sg.aws_security_group_rule.egress_rules[0]
module.app_security_group.module.sg.aws_security_group_rule.ingress_rules[0]
module.app_security_group.module.sg.aws_security_group_rule.ingress_rules[1]
module.app_security_group.module.sg.aws_security_group_rule.ingress_rules[2]
module.app_security_group.module.sg.aws_security_group_rule.ingress_rules[3]
module.app_security_group.module.sg.aws_security_group_rule.ingress_with_self[0]
module.elb_http.module.elb.aws_elb.this[0]
module.elb_http.module.elb_attachment.aws_elb_attachment.this[0]
module.elb_http.module.elb_attachment.aws_elb_attachment.this[1]
module.lb_security_group.module.sg.aws_security_group.this_name_prefix[0]
module.lb_security_group.module.sg.aws_security_group_rule.egress_rules[0]
module.lb_security_group.module.sg.aws_security_group_rule.ingress_rules[0]
module.lb_security_group.module.sg.aws_security_group_rule.ingress_rules[1]
module.lb_security_group.module.sg.aws_security_group_rule.ingress_rules[2]
module.lb_security_group.module.sg.aws_security_group_rule.ingress_rules[3]
module.lb_security_group.module.sg.aws_security_group_rule.ingress_with_self[0]

xescugc · 2021-02-23T10:49:10Z

Ok I see the issue.

The logic right now does not separate resources of different modules, so:

module.lb.aws_security_group.name
module.vpc.aws_security_group.name

Are understood as aws_security_group.name so it says that those are repeated.

xescugc · 2021-03-09T15:39:07Z

Merged the fix to master.

PePoDev · 2021-06-21T08:33:39Z

@xescugc I update to version 0.6.6 And still got Error: with canonical "module.ec2_ubuntu_a.aws_instance.this": graph node already exists with command inframap generate terraform.tfstate | dot -Tpng > graph.png

$ terraform state list
aws_key_pair.key_pair
local_file.ansible_inventory
local_file.private_key
null_resource.ansible_installation
tls_private_key.private_key
module.ec2_bastion.aws_instance.this[0]
module.ec2_redhat_a.aws_instance.this[0]
module.ec2_redhat_b.aws_instance.this[0]
module.ec2_ubuntu_a[0].aws_instance.this[0]
module.ec2_ubuntu_a[1].aws_instance.this[0]
module.ec2_ubuntu_b[0].aws_instance.this[0]
module.ec2_ubuntu_b[1].aws_instance.this[0]
module.ssh_sg.aws_security_group.this_name_prefix[0]
module.ssh_sg.aws_security_group_rule.ingress_rules[0]
module.vpc.aws_internet_gateway.this[0]
module.vpc.aws_route.public_internet_gateway[0]
module.vpc.aws_route_table.private[0]
module.vpc.aws_route_table.private[1]
module.vpc.aws_route_table.public[0]
module.vpc.aws_route_table_association.private[0]
module.vpc.aws_route_table_association.private[1]
module.vpc.aws_route_table_association.public[0]
module.vpc.aws_subnet.private[0]
module.vpc.aws_subnet.private[1]
module.vpc.aws_subnet.public[0]
module.vpc.aws_vpc.this[0]

xescugc · 2021-06-22T14:23:51Z

Ok I see the error.

We are not (I'll have to check how) considering a module used multiple times, for what I see in your case you have the ec2_ubuntu_a used multiple times.

I will check and find a way to identify it :).

BTW could you run inframap prune terraform.tfstate? This will STDOUT a new TFState with only the information we need (removing private information). You could also add a --canonicals which will also anonimize the resource names (which I think it's not needed on your case as you already share them :) ).

If with the new TFState still fails for the same reason then, if it's ok with you, you could share it so I actually write a test case for it :). Only if it's ok for you ofc.

PePoDev · 2021-06-23T05:39:37Z

{
  "version": 4,
  "terraform_version": "0.15.3",
  "serial": 899,
  "lineage": "d170e278-4de7-37d9-5083-fd901247cebd",
  "outputs": {},
  "resources": [
    {
      "module": "module.all_sg",
      "mode": "managed",
      "type": "aws_security_group",
      "name": "this_name_prefix",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "egress": [],
            "id": "sg-0987114dd4c1ff786",
            "ingress": []
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.vpc.aws_vpc.this"
          ],
          "create_before_destroy": true
        }
      ]
    },
    {
      "module": "module.all_sg",
      "mode": "managed",
      "type": "aws_security_group_rule",
      "name": "egress_rules",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 2,
          "attributes": {
            "id": "sgrule-3777339840",
            "security_group_id": "sg-0987114dd4c1ff786",
            "source_security_group_id": null
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_vpc.this"
          ]
        }
      ]
    },
    {
      "module": "module.all_sg",
      "mode": "managed",
      "type": "aws_security_group_rule",
      "name": "ingress_rules",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 2,
          "attributes": {
            "id": "sgrule-1734591447",
            "security_group_id": "sg-0987114dd4c1ff786",
            "source_security_group_id": null
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_vpc.this"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_openvpn.module.bastion_sg",
      "mode": "managed",
      "type": "aws_security_group",
      "name": "this_name_prefix",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "egress": [],
            "id": "sg-0004cc061bc236c1c",
            "ingress": []
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.vpc.aws_vpc.this"
          ],
          "create_before_destroy": true
        }
      ]
    },
    {
      "module": "module.ec2_openvpn.module.bastion_sg",
      "mode": "managed",
      "type": "aws_security_group_rule",
      "name": "egress_rules",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 2,
          "attributes": {
            "id": "sgrule-82634802",
            "security_group_id": "sg-0004cc061bc236c1c",
            "source_security_group_id": null
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.ec2_openvpn.module.bastion_sg.aws_security_group.this",
            "module.ec2_openvpn.module.bastion_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_vpc.this"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_openvpn.module.bastion_sg",
      "mode": "managed",
      "type": "aws_security_group_rule",
      "name": "ingress_rules",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 2,
          "attributes": {
            "id": "sgrule-96428233",
            "security_group_id": "sg-0004cc061bc236c1c",
            "source_security_group_id": null
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.ec2_openvpn.module.bastion_sg.aws_security_group.this",
            "module.ec2_openvpn.module.bastion_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_vpc.this"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_openvpn.module.ec2_bastion",
      "mode": "managed",
      "type": "aws_instance",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "id": "i-0a21b3e118e0ba2ef"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "aws_key_pair.key_pair",
            "module.ec2_openvpn.module.bastion_sg.aws_security_group.this",
            "module.ec2_openvpn.module.bastion_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_subnet.public",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this",
            "tls_private_key.private_key"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_redhat_a",
      "mode": "managed",
      "type": "aws_instance",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "id": "i-05f9a6abce16ab928"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "aws_key_pair.key_pair",
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_subnet.private",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this",
            "tls_private_key.private_key"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_redhat_b",
      "mode": "managed",
      "type": "aws_instance",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "id": "i-0befa657413af7c25"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "aws_key_pair.key_pair",
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_subnet.private",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this",
            "tls_private_key.private_key"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_ubuntu_a[0]",
      "mode": "managed",
      "type": "aws_instance",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "id": "i-01cde81471bc7715c"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "aws_key_pair.key_pair",
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_subnet.private",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this",
            "tls_private_key.private_key"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_ubuntu_a[1]",
      "mode": "managed",
      "type": "aws_instance",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "id": "i-0bbdaaaca5cf61ef5"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "aws_key_pair.key_pair",
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_subnet.private",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this",
            "tls_private_key.private_key"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_ubuntu_b[0]",
      "mode": "managed",
      "type": "aws_instance",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "id": "i-09f7c0b262ec08dfa"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "aws_key_pair.key_pair",
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_subnet.private",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this",
            "tls_private_key.private_key"
          ]
        }
      ]
    },
    {
      "module": "module.ec2_ubuntu_b[1]",
      "mode": "managed",
      "type": "aws_instance",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 1,
          "attributes": {
            "id": "i-0df31351fc10da682"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "aws_key_pair.key_pair",
            "module.all_sg.aws_security_group.this",
            "module.all_sg.aws_security_group.this_name_prefix",
            "module.vpc.aws_subnet.private",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this",
            "tls_private_key.private_key"
          ]
        }
      ]
    },
    {
      "module": "module.vpc",
      "mode": "managed",
      "type": "aws_eip",
      "name": "nat",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 0,
          "attributes": {
            "id": "eipalloc-0353cd2b1a2c0ed66"
          },
          "sensitive_attributes": []
        }
      ]
    },
    {
      "module": "module.vpc",
      "mode": "managed",
      "type": "aws_internet_gateway",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 0,
          "attributes": {
            "id": "igw-0e982a7cdfc39d2ff"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this"
          ]
        }
      ]
    },
    {
      "module": "module.vpc",
      "mode": "managed",
      "type": "aws_nat_gateway",
      "name": "this",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "index_key": 0,
          "schema_version": 0,
          "attributes": {
            "id": "nat-0d46ed83250f7d846"
          },
          "sensitive_attributes": [],
          "dependencies": [
            "module.vpc.aws_eip.nat",
            "module.vpc.aws_internet_gateway.this",
            "module.vpc.aws_subnet.public",
            "module.vpc.aws_vpc.this",
            "module.vpc.aws_vpc_ipv4_cidr_block_association.this"
          ]
        }
      ]
    }
  ]
}

PePoDev · 2021-07-02T11:19:54Z

I tested on branch fg-103 and It's working.

Thank you for putting up the extra time to reach our targets. Good job!

xescugc self-assigned this Feb 23, 2021

xescugc added the bug Something isn't working label Feb 23, 2021

xescugc added this to To do in InfraMap Feb 23, 2021

xescugc moved this from To do to In progress in InfraMap Mar 2, 2021

xescugc mentioned this issue Mar 2, 2021

generate: Add support for multiple modules having same resource names #111

Merged

xescugc closed this as completed in #111 Mar 9, 2021

xescugc moved this from In progress to Done in InfraMap Mar 9, 2021

xescugc reopened this Jun 22, 2021

xescugc moved this from Done to In progress in InfraMap Jun 22, 2021

xescugc mentioned this issue Jul 1, 2021

generate: Fix issue with repeated modules #147

Merged

PePoDev closed this as completed Jul 2, 2021

xescugc moved this from In progress to Done in InfraMap Jul 5, 2021

xescugc mentioned this issue Jul 5, 2021

Not Work With TFE State File #136

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error: with canonical "aws_security_group.this_name_prefix": graph node already exists #103

Error: with canonical "aws_security_group.this_name_prefix": graph node already exists #103

PePoDev commented Feb 20, 2021

rpgd60 commented Feb 22, 2021 •

edited

xescugc commented Feb 23, 2021

xescugc commented Mar 9, 2021

PePoDev commented Jun 21, 2021

xescugc commented Jun 22, 2021

PePoDev commented Jun 23, 2021

PePoDev commented Jul 2, 2021

Error: with canonical "aws_security_group.this_name_prefix": graph node already exists #103

Error: with canonical "aws_security_group.this_name_prefix": graph node already exists #103

Comments

PePoDev commented Feb 20, 2021

rpgd60 commented Feb 22, 2021 • edited

xescugc commented Feb 23, 2021

xescugc commented Mar 9, 2021

PePoDev commented Jun 21, 2021

xescugc commented Jun 22, 2021

PePoDev commented Jun 23, 2021

PePoDev commented Jul 2, 2021

rpgd60 commented Feb 22, 2021 •

edited