Unable to add multiple network interfaces to ec2 instance

[jcrawshaw20]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

terraform: v0.12.24
provider.aws: v2.62.0

Affected Resource(s)

• aws_instance
• aws_network_interface
• aws_security_group
• aws_eip

Terraform Configuration Files

resource "aws_network_interface" "management" {
  subnet_id       = data.aws_subnet.management_subnet.id
  security_groups = [aws_security_group.management.id, aws_security_group.external.id]
}

resource "aws_network_interface" "external" {
  subnet_id         = data.aws_subnet.external_subnet.id
  private_ips_count = var.virtual_server_count
  security_groups   = [aws_security_group.management.id, aws_security_group.external.id]
}

resource "aws_security_group" "management" {
  name_prefix = "f5-management"
  vpc_id      = data.aws_vpc.vpc.id

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = var.management_security_group_ingress_cidrs
    self        = true
  }

  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = var.management_security_group_ingress_cidrs
  }

}

resource "aws_security_group" "external" {
  name_prefix = "f5-external"
  vpc_id      = data.aws_vpc.vpc.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port   = 443
    to_port     = 443
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  ingress {
    from_port = 4353
    to_port   = 4353
    protocol  = "tcp"
    self      = true
  }

  ingress {
    from_port = 1026
    to_port   = 1026
    protocol  = "udp"
    self      = true
  }

}

resource "aws_instance" "big_ip" {
  ami           = data.aws_ami.big_ip_image.id
  instance_type = var.instance_type
  key_name      = var.ssh_key_name
  tags = {
    Name = join("-", ["Big-IP", var.instance_identifier])
  }
  subnet_id = data.aws_subnet.management_subnet.id

  network_interface {
    device_index         = 0
    network_interface_id = aws_network_interface.management.id
  }

  network_interface {
    device_index         = 1
    network_interface_id = aws_network_interface.external.id
  }

  vpc_security_group_ids = [
    aws_security_group.management.id, aws_security_group.external.id
  ]
}

resource "aws_eip" "management" {
  network_interface         = aws_network_interface.management.id
  associate_with_private_ip = aws_network_interface.management.private_ip
}

data "aws_network_interface" "external" {
  id = aws_network_interface.external.id
}

resource "aws_eip" "virtual_server" {
  count                     = var.virtual_server_count
  network_interface         = aws_network_interface.external.id
  associate_with_private_ip = sort(setsubtract(data.aws_network_interface.external.private_ips, [data.aws_network_interface.external.private_ip]))[count.index]
}

Debug Output

    The following problems may be the cause of any confusing errors from downstream operations:
cty.True, "to_port":cty.NumberIntVal(443)})}) nor prior value cty.SetVal([]cty.Value{cty.ObjectVal(map[string]cty.Value{"cidr_blocks":cty.ListVal([]cty.Value{cty.StringVal("178.62.118.230/32")}), "description":cty.StringVal(""), "from_port":cty.NumberIntVal(22), "ipv6_cidr_blocks":cty.ListValEmpty(cty.String), "prefix_list_ids":cty.ListValEmpty(cty.String), "protocol":cty.StringVal("tcp"), "security_groups":cty.SetValEmpty(cty.String), "self":cty.False, "to_port":cty.NumberIntVal(22)}), cty.ObjectVal(map[string]cty.Value{"cidr_blocks":cty.ListVal([]cty.Value{cty.StringVal("178.62.118.230/32")}), "description":cty.StringVal(""), "from_port":cty.NumberIntVal(443), "ipv6_cidr_blocks":cty.ListValEmpty(cty.String), "prefix_list_ids":cty.ListValEmpty(cty.String), "protocol":cty.StringVal("tcp"), "security_groups":cty.SetValEmpty(cty.String), "self":cty.True, "to_port":cty.NumberIntVal(443)})})
2020/05/15 11:27:12 [WARN] Provider "registry.terraform.io/-/aws" produced an invalid plan for module.test.aws_security_group.external, but we are tolerating it because it is using the legacy plugin SDK.
    The following problems may be the cause of any confusing errors from downstream operations:
string]cty.Value{"cidr_blocks":cty.ListVal([]cty.Value{cty.StringVal("0.0.0.0/0")}), "description":cty.StringVal(""), "from_port":cty.NumberIntVal(80), "ipv6_cidr_blocks":cty.ListValEmpty(cty.String), "prefix_list_ids":cty.ListValEmpty(cty.String), "protocol":cty.StringVal("tcp"), "security_groups":cty.SetValEmpty(cty.String), "self":cty.False, "to_port":cty.NumberIntVal(80)}), cty.ObjectVal(map[string]cty.Value{"cidr_blocks":cty.ListValEmpty(cty.String), "description":cty.StringVal(""), "from_port":cty.NumberIntVal(1026), "ipv6_cidr_blocks":cty.ListValEmpty(cty.String), "prefix_list_ids":cty.ListValEmpty(cty.String), "protocol":cty.StringVal("udp"), "security_groups":cty.SetValEmpty(cty.String), "self":cty.True, "to_port":cty.NumberIntVal(1026)}), cty.ObjectVal(map[string]cty.Value{"cidr_blocks":cty.ListValEmpty(cty.String), "description":cty.StringVal(""), "from_port":cty.NumberIntVal(4353), "ipv6_cidr_blocks":cty.ListValEmpty(cty.String), "prefix_list_ids":cty.ListValEmpty(cty.String), "protocol":cty.StringVal("tcp"), "security_groups":cty.SetValEmpty(cty.String), "self":cty.True, "to_port":cty.NumberIntVal(4353)})})
2020/05/15 11:27:12 [DEBUG] ReferenceTransformer: "module.test.aws_network_interface.management" references: []
2020/05/15 11:27:12 [DEBUG] ReferenceTransformer: "module.test.aws_network_interface.external" references: []
2020/05/15 11:27:12 [DEBUG] ReferenceTransformer: "module.test.aws_eip.management" references: []
2020/05/15 11:27:12 [DEBUG] ReferenceTransformer: "module.test.aws_instance.big_ip" references: []
2020/05/15 11:27:12 [DEBUG] ReferenceTransformer: "module.test.data.aws_network_interface.external" references: []
2020/05/15 11:27:12 [DEBUG] ReferenceTransformer: "module.test.aws_eip.virtual_server[0]" references: []
2020/05/15 11:27:12 [DEBUG] ReferenceTransformer: "module.test.aws_eip.virtual_server[1]" references: []
2020/05/15 11:27:12 [ERROR] module.test: eval: *terraform.EvalDiff, err: "network_interface": conflicts with subnet_id
2020/05/15 11:27:12 [ERROR] module.test: eval: *terraform.EvalSequence, err: "network_interface": conflicts with subnet_id

Error: "network_interface": conflicts with subnet_id

  on ../big_ip.tf line 11, in resource "aws_instance" "big_ip":
  11: resource "aws_instance" "big_ip" {


2020-05-15T11:27:12.479+0100 [DEBUG] plugin: plugin process exited: path=/mnt/c/Users/JoshuaRushton-Crawsh/terraform-modules/f5/auden-f5-VIP/test/.terraform/plugins/linux_amd64/terraform-provider-aws_v2.62.0_x4 pid=3414
2020-05-15T11:27:12.479+0100 [DEBUG] plugin: plugin exited

Panic Output

Error: "network_interface": conflicts with subnet_id

Expected Behavior

Ec2 instance starts up with 2 network interfaces

Actual Behavior

Instance isn't created with that error message

Steps to Reproduce

1. terraform apply

Important Factoids

References

• #0000

[rick-masters]

Specifying both network interface and subnet is not supported by AWS.
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html

[dilshans2k]

Specifying both network interface and subnet is not supported by AWS. https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html

Could you please highlight, where it is mentioned..

[rick-masters]

@dilshans2k
Please see the SubnetId parameter.

SubnetId
The ID of the subnet to launch the instance into.

If you specify a network interface, you must specify any subnets as part of the network interface.

Type: String

Required: No

It could be clearer by adding " and you must not use this parameter." but that is what it is trying to say.
Having both would be redundant and possibly in conflict. There is no need for both.

There are other reports that this is not allowed:

aws/aws-sdk-php#231

Network interfaces and an instance-level subnet ID may not be specified on the same request

https://stackoverflow.com/questions/42995644/network-interface-and-an-instance-level-subnet-id-may-not-be-specified-on-same

https://stackoverflow.com/questions/71556507/network-interfaces-and-an-instance-level-private-ip-address-may-not-be-specified