New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using count parameter w/ a split in aws_security_group_rule, receive parsing error "strconv.ParseInt" .. " invalid syntax" #3888

Closed
timothykimball opened this Issue Nov 12, 2015 · 39 comments

Comments

Projects
None yet
@timothykimball

timothykimball commented Nov 12, 2015

This is using terraform v0.6.5

The following module is used:

resource "aws_security_group_rule" "rule" {
    count = "${length(split(",",var.ids))}"
    type = "${var.type}"
    from_port = "${var.port}"
    to_port = "${var.port}"
    protocol = "tcp"
    security_group_id = "${var.sg_id}"
    source_security_group_id = "${element(split(",",var.ids), count.index)}"
}

and when I run in conjunction with the wider project:

Errors:

  * strconv.ParseInt: parsing "${length(split(\",\",var.ids))}": invalid syntax

If I rewrite

count = "${length(split(",",var.ids))}"

as

count = "${length(split(",","a,b,c"))}"*

it works...

@timothykimball

This comment has been minimized.

Show comment
Hide comment
@timothykimball

timothykimball Nov 12, 2015

Also: Managed to get the "${length(split(",",var.ids))}" printed to string. The value was 4 (as expected).

timothykimball commented Nov 12, 2015

Also: Managed to get the "${length(split(",",var.ids))}" printed to string. The value was 4 (as expected).

@Deserved

This comment has been minimized.

Show comment
Hide comment

Deserved commented Nov 12, 2015

@jen20

This comment has been minimized.

Show comment
Hide comment
@jen20

jen20 Nov 12, 2015

Contributor

Hi @timothykimball. I'm trying to reproduce this on the master branch at the moment, using the following minimal .tf file:

provider "aws" {
    region = "us-west-2"
}

variable "ids" {
    default = "sg-12345,sg-67890,sg-34567"
}

resource "aws_security_group_rule" "rule" {
    count = "${length(split(",",var.ids))}"
    type = "ingress"
    from_port = "80"
    to_port = "80"
    protocol = "tcp"
    security_group_id = "sg-123456789"
    source_security_group_id = "${element(split(",",var.ids), count.index)}"
}

Running terraform plan gives me the following output:

Refreshing Terraform state prior to plan...


The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

+ aws_security_group_rule.rule.0


    from_port:                "" => "80"
    protocol:                 "" => "tcp"
    security_group_id:        "" => "sg-123456789"
    self:                     "" => "0"
    source_security_group_id: "" => "sg-12345"
    to_port:                  "" => "80"
    type:                     "" => "ingress"

+ aws_security_group_rule.rule.1
    from_port:                "" => "80"
    protocol:                 "" => "tcp"
    security_group_id:        "" => "sg-123456789"
    self:                     "" => "0"
    source_security_group_id: "" => "sg-67890"
    to_port:                  "" => "80"
    type:                     "" => "ingress"

+ aws_security_group_rule.rule.2
    from_port:                "" => "80"
    protocol:                 "" => "tcp"
    security_group_id:        "" => "sg-123456789"
    self:                     "" => "0"
    source_security_group_id: "" => "sg-34567"
    to_port:                  "" => "80"
    type:                     "" => "ingress"


Plan: 3 to add, 0 to change, 0 to destroy.

It's possible that this was fixed between v0.6.5 and now though, since the parser has been replaced on master. Does the latest released version also exhibit this behaviour?

Contributor

jen20 commented Nov 12, 2015

Hi @timothykimball. I'm trying to reproduce this on the master branch at the moment, using the following minimal .tf file:

provider "aws" {
    region = "us-west-2"
}

variable "ids" {
    default = "sg-12345,sg-67890,sg-34567"
}

resource "aws_security_group_rule" "rule" {
    count = "${length(split(",",var.ids))}"
    type = "ingress"
    from_port = "80"
    to_port = "80"
    protocol = "tcp"
    security_group_id = "sg-123456789"
    source_security_group_id = "${element(split(",",var.ids), count.index)}"
}

Running terraform plan gives me the following output:

Refreshing Terraform state prior to plan...


The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

+ aws_security_group_rule.rule.0


    from_port:                "" => "80"
    protocol:                 "" => "tcp"
    security_group_id:        "" => "sg-123456789"
    self:                     "" => "0"
    source_security_group_id: "" => "sg-12345"
    to_port:                  "" => "80"
    type:                     "" => "ingress"

+ aws_security_group_rule.rule.1
    from_port:                "" => "80"
    protocol:                 "" => "tcp"
    security_group_id:        "" => "sg-123456789"
    self:                     "" => "0"
    source_security_group_id: "" => "sg-67890"
    to_port:                  "" => "80"
    type:                     "" => "ingress"

+ aws_security_group_rule.rule.2
    from_port:                "" => "80"
    protocol:                 "" => "tcp"
    security_group_id:        "" => "sg-123456789"
    self:                     "" => "0"
    source_security_group_id: "" => "sg-34567"
    to_port:                  "" => "80"
    type:                     "" => "ingress"


Plan: 3 to add, 0 to change, 0 to destroy.

It's possible that this was fixed between v0.6.5 and now though, since the parser has been replaced on master. Does the latest released version also exhibit this behaviour?

@jwieringa

This comment has been minimized.

Show comment
Hide comment
@jwieringa

jwieringa Dec 2, 2015

Contributor

I am also having this issue and my configs which are similar to @jen20's do work. For myself. I've been able to reproduce the issue by passing in values via interpolation in a module.

Assuming a module which looks like:

variable "vpc_id" {}
variable "sg_ids" {}

resource "aws_security_group" "test_3888" {
  name        = "test-3888"
  description = "Testing https://github.com/hashicorp/terraform/issues/3888"
  vpc_id      = "${var.vpc_id}"
}

resource "aws_security_group_rule" "rule" {
  count                    = "${length(split(",", var.sg_ids))}"
  type                     = "ingress"
  from_port                = "80"
  to_port                  = "80"
  protocol                 = "tcp"
  security_group_id        = "${aws_security_group.test_3888.id}"
  source_security_group_id = "${element(split(",", var.sg_ids), count.index)}"
}

This works:

module "test-count-01" {
  source = "./test-3888"
  sg_ids = "sg-e6183780"
  vpc_id = "${var.vpc_id}"
}

This does not work:

resource "aws_security_group" "sg_01" {
  name        = "test-3888"
  description = "Testing https://github.com/hashicorp/terraform/issues/3888"
  vpc_id      = "${var.vpc_id}"
}

module "test-count-08" {
  source = "./test-3888"
  sg_ids = "${aws_security_group.sg_01.id}"
  vpc_id = "${var.vpc_id}"
}

With the error message:

Errors:

  * strconv.ParseInt: parsing "${length(split(\",\", var.sg_ids))}": invalid syntax

Version:

$ terraform -v
Terraform v0.6.8
Contributor

jwieringa commented Dec 2, 2015

I am also having this issue and my configs which are similar to @jen20's do work. For myself. I've been able to reproduce the issue by passing in values via interpolation in a module.

Assuming a module which looks like:

variable "vpc_id" {}
variable "sg_ids" {}

resource "aws_security_group" "test_3888" {
  name        = "test-3888"
  description = "Testing https://github.com/hashicorp/terraform/issues/3888"
  vpc_id      = "${var.vpc_id}"
}

resource "aws_security_group_rule" "rule" {
  count                    = "${length(split(",", var.sg_ids))}"
  type                     = "ingress"
  from_port                = "80"
  to_port                  = "80"
  protocol                 = "tcp"
  security_group_id        = "${aws_security_group.test_3888.id}"
  source_security_group_id = "${element(split(",", var.sg_ids), count.index)}"
}

This works:

module "test-count-01" {
  source = "./test-3888"
  sg_ids = "sg-e6183780"
  vpc_id = "${var.vpc_id}"
}

This does not work:

resource "aws_security_group" "sg_01" {
  name        = "test-3888"
  description = "Testing https://github.com/hashicorp/terraform/issues/3888"
  vpc_id      = "${var.vpc_id}"
}

module "test-count-08" {
  source = "./test-3888"
  sg_ids = "${aws_security_group.sg_01.id}"
  vpc_id = "${var.vpc_id}"
}

With the error message:

Errors:

  * strconv.ParseInt: parsing "${length(split(\",\", var.sg_ids))}": invalid syntax

Version:

$ terraform -v
Terraform v0.6.8
@mberhault

This comment has been minimized.

Show comment
Hide comment
@mberhault

mberhault Dec 14, 2015

I seem to hit a similar parse error on occasion. However, I'm not having as much luck reproducing it:

terraform destroy --force --var=num_instances=0 --var=example_block_writer_instances=0 --var=key_name=cockroach-marc
 null_resource.cockroach-runner.1: Refreshing state... (ID: 4045808629333585387)
 null_resource.cockroach-runner.4: Refreshing state... (ID: 8276708105864651955)
 null_resource.cockroach-runner: Refreshing state... (ID: 244252520158716368)
 null_resource.cockroach-runner.3: Refreshing state... (ID: 8595918747006064252)
 null_resource.cockroach-runner.2: Refreshing state... (ID: 1125939317577129625)
 aws_instance.example_block_writer.1: Refreshing state... (ID: i-f0428241)
 aws_instance.cockroach.4: Refreshing state... (ID: i-564181e7)
 aws_instance.example_block_writer.4: Refreshing state... (ID: i-f2428243)
 aws_instance.cockroach: Refreshing state... (ID: i-184181a9)
 aws_instance.example_block_writer.2: Refreshing state... (ID: i-20418191)
 aws_instance.cockroach.3: Refreshing state... (ID: i-1f4181ae)
 aws_instance.cockroach.2: Refreshing state... (ID: i-80428231)
 aws_instance.cockroach.1: Refreshing state... (ID: i-574181e6)
 aws_instance.example_block_writer.3: Refreshing state... (ID: i-7d4181cc)
 aws_instance.example_block_writer: Refreshing state... (ID: i-f3428242)
 aws_security_group.default: Refreshing state... (ID: sg-0aa1be6c)
 null_resource.cockroach-initializer: Refreshing state... (ID: 7365069503254632587)
 aws_elb.elb: Refreshing state... (ID: cockroach-marc-elb)
 template_file.supervisor: Refreshing state... (ID: e15d5b0a8f0914f8432637b05db51be395037086dcbf521c8b7d16f17d793070)
 Error creating plan: 1 error(s) occurred:

* strconv.ParseInt: parsing "${var.example_block_writer_instances}": invalid syntax

The relevant config:

resource "aws_instance" "example_block_writer" {
  tags {
    Name = "${var.key_name}-block-writer"
  }

  ami = "${var.aws_ami_id}"
  availability_zone = "${var.aws_availability_zone}"
  instance_type = "${var.aws_instance_type}"
  security_groups = ["${aws_security_group.default.name}"]
  key_name = "${var.key_name}"
  count = "${var.example_block_writer_instances}"

 ... (connection, file, and remote-exec here)

The fact that the strconv error shows the variable name as opposed to its value seems to indicate that it wasn't interpolated properly. This comes from:

go/src/strconv/atoi.go:
func (e *NumError) Error() string {
  return "strconv." + e.Func + ": " + "parsing " + Quote(e.Num) + ": " + e.Err.Error()
}

I've seen this on 0.6.6, 0.6.7 and 0.6.8.

mberhault commented Dec 14, 2015

I seem to hit a similar parse error on occasion. However, I'm not having as much luck reproducing it:

terraform destroy --force --var=num_instances=0 --var=example_block_writer_instances=0 --var=key_name=cockroach-marc
 null_resource.cockroach-runner.1: Refreshing state... (ID: 4045808629333585387)
 null_resource.cockroach-runner.4: Refreshing state... (ID: 8276708105864651955)
 null_resource.cockroach-runner: Refreshing state... (ID: 244252520158716368)
 null_resource.cockroach-runner.3: Refreshing state... (ID: 8595918747006064252)
 null_resource.cockroach-runner.2: Refreshing state... (ID: 1125939317577129625)
 aws_instance.example_block_writer.1: Refreshing state... (ID: i-f0428241)
 aws_instance.cockroach.4: Refreshing state... (ID: i-564181e7)
 aws_instance.example_block_writer.4: Refreshing state... (ID: i-f2428243)
 aws_instance.cockroach: Refreshing state... (ID: i-184181a9)
 aws_instance.example_block_writer.2: Refreshing state... (ID: i-20418191)
 aws_instance.cockroach.3: Refreshing state... (ID: i-1f4181ae)
 aws_instance.cockroach.2: Refreshing state... (ID: i-80428231)
 aws_instance.cockroach.1: Refreshing state... (ID: i-574181e6)
 aws_instance.example_block_writer.3: Refreshing state... (ID: i-7d4181cc)
 aws_instance.example_block_writer: Refreshing state... (ID: i-f3428242)
 aws_security_group.default: Refreshing state... (ID: sg-0aa1be6c)
 null_resource.cockroach-initializer: Refreshing state... (ID: 7365069503254632587)
 aws_elb.elb: Refreshing state... (ID: cockroach-marc-elb)
 template_file.supervisor: Refreshing state... (ID: e15d5b0a8f0914f8432637b05db51be395037086dcbf521c8b7d16f17d793070)
 Error creating plan: 1 error(s) occurred:

* strconv.ParseInt: parsing "${var.example_block_writer_instances}": invalid syntax

The relevant config:

resource "aws_instance" "example_block_writer" {
  tags {
    Name = "${var.key_name}-block-writer"
  }

  ami = "${var.aws_ami_id}"
  availability_zone = "${var.aws_availability_zone}"
  instance_type = "${var.aws_instance_type}"
  security_groups = ["${aws_security_group.default.name}"]
  key_name = "${var.key_name}"
  count = "${var.example_block_writer_instances}"

 ... (connection, file, and remote-exec here)

The fact that the strconv error shows the variable name as opposed to its value seems to indicate that it wasn't interpolated properly. This comes from:

go/src/strconv/atoi.go:
func (e *NumError) Error() string {
  return "strconv." + e.Func + ": " + "parsing " + Quote(e.Num) + ": " + e.Err.Error()
}

I've seen this on 0.6.6, 0.6.7 and 0.6.8.

@deftflux

This comment has been minimized.

Show comment
Hide comment
@deftflux

deftflux Jan 5, 2016

Contributor

I too ran into this same issue. I am trying to use null_resource to apply some file and remote-exec provisioners to a set of resources defined in another module as a way to reuse certain scripts across different resources. I was using code copied from here, except replacing references to aws_instance with input variables. When setting count = "${length(split(",", var.hosts))}" I got the same parsing error as mentioned here.

I did some troubleshooting, however, and I was able to find a workaround and a probable cause.

As a workaround, I passed a second input variable for the count in addition to the joined list, and in the referencing module, I specified a constant value for the variable. Why would this work and not other interpolated strings?

It seems that it is interpolating the string either way, but in the former case, the value at that point in time is not a valid integer. That is because the value of count must be known ahead of time, but the value of the variable that split depends on is not known until later. It seems that Terraform puts "<computed>" in its place temporarily until the value is known, and <computed> is not a valid integer.

I would suggest two improvements to address this issue:

  1. Improve the error message in this situation. Show the value after interpolation that it is attempting to parse as an integer in addition to the original supplied value. If the reason it cannot be parsed to an integer is because the actual value cannot be computed at this time, explain this and suggest alternatives.
  2. Provide a way to infer the count ahead of time. In this case, there is no reason why the count cannot be known ahead of time. It is based on the number of aws_instance resources, which is known. You might do this in one of a few ways:
    • Make length smarter; when it refers to a list, calculate what the length of that list would be regardless of the values that it would contain. In addition, join and split would have to be similarly smarter.
    • Provide a way to get the length of a collection without referring to a specific attribute. For example, instead of having to do length(aws_instance.foo.*.id) which depends upon id which is not known until later, something like aws_instance.foo.count could be used.
    • Add support for lists being passed through variables instead of just strings. This would avoid having to join and split the values. The count could then be inferred more easily.
Contributor

deftflux commented Jan 5, 2016

I too ran into this same issue. I am trying to use null_resource to apply some file and remote-exec provisioners to a set of resources defined in another module as a way to reuse certain scripts across different resources. I was using code copied from here, except replacing references to aws_instance with input variables. When setting count = "${length(split(",", var.hosts))}" I got the same parsing error as mentioned here.

I did some troubleshooting, however, and I was able to find a workaround and a probable cause.

As a workaround, I passed a second input variable for the count in addition to the joined list, and in the referencing module, I specified a constant value for the variable. Why would this work and not other interpolated strings?

It seems that it is interpolating the string either way, but in the former case, the value at that point in time is not a valid integer. That is because the value of count must be known ahead of time, but the value of the variable that split depends on is not known until later. It seems that Terraform puts "<computed>" in its place temporarily until the value is known, and <computed> is not a valid integer.

I would suggest two improvements to address this issue:

  1. Improve the error message in this situation. Show the value after interpolation that it is attempting to parse as an integer in addition to the original supplied value. If the reason it cannot be parsed to an integer is because the actual value cannot be computed at this time, explain this and suggest alternatives.
  2. Provide a way to infer the count ahead of time. In this case, there is no reason why the count cannot be known ahead of time. It is based on the number of aws_instance resources, which is known. You might do this in one of a few ways:
    • Make length smarter; when it refers to a list, calculate what the length of that list would be regardless of the values that it would contain. In addition, join and split would have to be similarly smarter.
    • Provide a way to get the length of a collection without referring to a specific attribute. For example, instead of having to do length(aws_instance.foo.*.id) which depends upon id which is not known until later, something like aws_instance.foo.count could be used.
    • Add support for lists being passed through variables instead of just strings. This would avoid having to join and split the values. The count could then be inferred more easily.
@Gary-Armstrong

This comment has been minimized.

Show comment
Hide comment
@Gary-Armstrong

Gary-Armstrong Jan 11, 2016

Outputting the values which strconv.ParseInt has issue with would definitely help me. I'm having a basic version of this issue in an inconsistent manner, and multiple runs are only very slowly revealing bits of the pattern.

Just for the record, I'm seeing what is documented in #1373 but not every time! Since I don't know what the exact value is, nor whether it is cast as str or int (does Go do this?), getting to the root is very difficult.

Even better, the trace to logfile just stops on this error and nothing is recorded. Not even the strconv.ParseInt error.

Summary info for my case [v0.6.8]:
terraform.tfvars:
compute_count = 6
instances.tf:

resource "aws_instance" "compute" {
  # unquoted integer variable gives illegal char error
  #count = ${var.compute_count}
  count = "${var.compute_count}"
...

Plan it. Multiple CLI execs are run as fast as I can recall the command:

$ terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * strconv.ParseInt: parsing "${var.compute_count}": invalid syntax
$ terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * strconv.ParseInt: parsing "${var.compute_count}": invalid syntax
$ terraform plan
Refreshing Terraform state prior to plan...

null_resource.provision_head: Refreshing state... (ID: 8154872947017937993)
null_resource.provision_compute.0: Refreshing state... (ID: 5449962867436015499)
null_resource.provision_compute.4: Refreshing state... (ID: 2840353526046536871)
null_resource.provision_viewer: Refreshing state... (ID: 2333909372834370809)
null_resource.provision_compute.1: Refreshing state... (ID: 149251303276816687)
null_resource.provision_compute.2: Refreshing state... (ID: 5089136462066237229)
null_resource.provision_compute.3: Refreshing state... (ID: 737231409558745664)
null_resource.provision_compute.5: Refreshing state... (ID: 3993076988363024179)
aws_iam_role.wxmix_role: Refreshing state... (ID: wxmix-qa-us-east-1-role)
aws_iam_policy.wxmix_policy: Refreshing state... (ID: XXX-policy)
aws_vpc.main: Refreshing state... (ID: vpc-55d8ad31)
aws_iam_instance_profile.wxmix_profile: Refreshing state... (ID: wxmix-qa-us-east-1-profile)
aws_iam_policy_attachment.wxmix_policy-attach: Refreshing state... (ID: wxmix-qa-us-east-1-policy-attach)
aws_security_group.nat_sg: Refreshing state... (ID: sg-fd941784)
aws_internet_gateway.main: Refreshing state... (ID: igw-66a95002)
aws_security_group.wxmix_sg: Refreshing state... (ID: sg-fa941783)
aws_vpc_peering_connection.legacy_sun_vpc: Refreshing state... (ID: pcx-65c11b0c)
aws_subnet.private: Refreshing state... (ID: subnet-0545ac2f)
aws_security_group.admin_sg: Refreshing state... (ID: sg-f8941781)
aws_security_group_rule.admin_ingress_ICMP: Refreshing state... (ID: sgrule-771614868)
aws_security_group_rule.admin_ingress_22_legacyvpc: Refreshing state... (ID: sgrule-1668773577)
aws_security_group_rule.admin_egress_all: Refreshing state... (ID: sgrule-370978838)
aws_security_group_rule.admin_ingress_22_vpc: Refreshing state... (ID: sgrule-49088385)
aws_security_group_rule.admin_ingress_22_wfh_ip: Refreshing state... (ID: sgrule-879157753)
aws_security_group_rule.admin_ingress_22_offices: Refreshing state... (ID: sgrule-4037036570)
aws_route_table.public: Refreshing state... (ID: rtb-4de7cf29)
aws_security_group_rule.admin_ingress_22_wsi: Refreshing state... (ID: sgrule-1222866253)
aws_security_group_rule.nat_ingress_all: Refreshing state... (ID: sgrule-3917436300)
aws_security_group_rule.nat_egress_all: Refreshing state... (ID: sgrule-550358490)
aws_security_group_rule.wxmix_ftp: Refreshing state... (ID: sgrule-308021746)
aws_security_group_rule.wxmix_subnet: Refreshing state... (ID: sgrule-3099808515)
aws_security_group_rule.wxmix_perforce: Refreshing state... (ID: sgrule-380179889)
aws_security_group_rule.wxmix_postgres: Refreshing state... (ID: sgrule-2352848364)
aws_security_group_rule.wxmix_rmq: Refreshing state... (ID: sgrule-3698139287)
aws_security_group_rule.wxmix_http: Refreshing state... (ID: sgrule-575798803)
aws_security_group_rule.wxmix_ldm: Refreshing state... (ID: sgrule-2819647739)
aws_security_group_rule.wxmix_https: Refreshing state... (ID: sgrule-2302824115)
aws_main_route_table_association.a: Refreshing state... (ID: rtbassoc-2601a041)
Error refreshing state: 1 error(s) occurred:

* strconv.ParseInt: parsing "${var.head_count}": invalid syntax
gary.armstrong$ terraform plan
Refreshing Terraform state prior to plan...

null_resource.provision_head: Refreshing state... (ID: 8154872947017937993)
null_resource.provision_compute.1: Refreshing state... (ID: 149251303276816687)
null_resource.provision_compute.4: Refreshing state... (ID: 2840353526046536871)
null_resource.provision_compute.5: Refreshing state... (ID: 3993076988363024179)
null_resource.provision_viewer: Refreshing state... (ID: 2333909372834370809)
... [ SUCCEEDS ]

Gary-Armstrong commented Jan 11, 2016

Outputting the values which strconv.ParseInt has issue with would definitely help me. I'm having a basic version of this issue in an inconsistent manner, and multiple runs are only very slowly revealing bits of the pattern.

Just for the record, I'm seeing what is documented in #1373 but not every time! Since I don't know what the exact value is, nor whether it is cast as str or int (does Go do this?), getting to the root is very difficult.

Even better, the trace to logfile just stops on this error and nothing is recorded. Not even the strconv.ParseInt error.

Summary info for my case [v0.6.8]:
terraform.tfvars:
compute_count = 6
instances.tf:

resource "aws_instance" "compute" {
  # unquoted integer variable gives illegal char error
  #count = ${var.compute_count}
  count = "${var.compute_count}"
...

Plan it. Multiple CLI execs are run as fast as I can recall the command:

$ terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * strconv.ParseInt: parsing "${var.compute_count}": invalid syntax
$ terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * strconv.ParseInt: parsing "${var.compute_count}": invalid syntax
$ terraform plan
Refreshing Terraform state prior to plan...

null_resource.provision_head: Refreshing state... (ID: 8154872947017937993)
null_resource.provision_compute.0: Refreshing state... (ID: 5449962867436015499)
null_resource.provision_compute.4: Refreshing state... (ID: 2840353526046536871)
null_resource.provision_viewer: Refreshing state... (ID: 2333909372834370809)
null_resource.provision_compute.1: Refreshing state... (ID: 149251303276816687)
null_resource.provision_compute.2: Refreshing state... (ID: 5089136462066237229)
null_resource.provision_compute.3: Refreshing state... (ID: 737231409558745664)
null_resource.provision_compute.5: Refreshing state... (ID: 3993076988363024179)
aws_iam_role.wxmix_role: Refreshing state... (ID: wxmix-qa-us-east-1-role)
aws_iam_policy.wxmix_policy: Refreshing state... (ID: XXX-policy)
aws_vpc.main: Refreshing state... (ID: vpc-55d8ad31)
aws_iam_instance_profile.wxmix_profile: Refreshing state... (ID: wxmix-qa-us-east-1-profile)
aws_iam_policy_attachment.wxmix_policy-attach: Refreshing state... (ID: wxmix-qa-us-east-1-policy-attach)
aws_security_group.nat_sg: Refreshing state... (ID: sg-fd941784)
aws_internet_gateway.main: Refreshing state... (ID: igw-66a95002)
aws_security_group.wxmix_sg: Refreshing state... (ID: sg-fa941783)
aws_vpc_peering_connection.legacy_sun_vpc: Refreshing state... (ID: pcx-65c11b0c)
aws_subnet.private: Refreshing state... (ID: subnet-0545ac2f)
aws_security_group.admin_sg: Refreshing state... (ID: sg-f8941781)
aws_security_group_rule.admin_ingress_ICMP: Refreshing state... (ID: sgrule-771614868)
aws_security_group_rule.admin_ingress_22_legacyvpc: Refreshing state... (ID: sgrule-1668773577)
aws_security_group_rule.admin_egress_all: Refreshing state... (ID: sgrule-370978838)
aws_security_group_rule.admin_ingress_22_vpc: Refreshing state... (ID: sgrule-49088385)
aws_security_group_rule.admin_ingress_22_wfh_ip: Refreshing state... (ID: sgrule-879157753)
aws_security_group_rule.admin_ingress_22_offices: Refreshing state... (ID: sgrule-4037036570)
aws_route_table.public: Refreshing state... (ID: rtb-4de7cf29)
aws_security_group_rule.admin_ingress_22_wsi: Refreshing state... (ID: sgrule-1222866253)
aws_security_group_rule.nat_ingress_all: Refreshing state... (ID: sgrule-3917436300)
aws_security_group_rule.nat_egress_all: Refreshing state... (ID: sgrule-550358490)
aws_security_group_rule.wxmix_ftp: Refreshing state... (ID: sgrule-308021746)
aws_security_group_rule.wxmix_subnet: Refreshing state... (ID: sgrule-3099808515)
aws_security_group_rule.wxmix_perforce: Refreshing state... (ID: sgrule-380179889)
aws_security_group_rule.wxmix_postgres: Refreshing state... (ID: sgrule-2352848364)
aws_security_group_rule.wxmix_rmq: Refreshing state... (ID: sgrule-3698139287)
aws_security_group_rule.wxmix_http: Refreshing state... (ID: sgrule-575798803)
aws_security_group_rule.wxmix_ldm: Refreshing state... (ID: sgrule-2819647739)
aws_security_group_rule.wxmix_https: Refreshing state... (ID: sgrule-2302824115)
aws_main_route_table_association.a: Refreshing state... (ID: rtbassoc-2601a041)
Error refreshing state: 1 error(s) occurred:

* strconv.ParseInt: parsing "${var.head_count}": invalid syntax
gary.armstrong$ terraform plan
Refreshing Terraform state prior to plan...

null_resource.provision_head: Refreshing state... (ID: 8154872947017937993)
null_resource.provision_compute.1: Refreshing state... (ID: 149251303276816687)
null_resource.provision_compute.4: Refreshing state... (ID: 2840353526046536871)
null_resource.provision_compute.5: Refreshing state... (ID: 3993076988363024179)
null_resource.provision_viewer: Refreshing state... (ID: 2333909372834370809)
... [ SUCCEEDS ]
@justinclayton

This comment has been minimized.

Show comment
Hide comment
@justinclayton

justinclayton Jan 12, 2016

Contributor

We get this as well, even with today's master. Maybe 20% of the plans produce this error. Almost always with count = "${var.count}" in our case, which is quite simple as far as interpolation goes.

Contributor

justinclayton commented Jan 12, 2016

We get this as well, even with today's master. Maybe 20% of the plans produce this error. Almost always with count = "${var.count}" in our case, which is quite simple as far as interpolation goes.

@colout

This comment has been minimized.

Show comment
Hide comment
@colout

colout Jan 14, 2016

I'm getting this error as well. It seems to have something to do with how the delimited list is generated. I can get this to work as an output variable, but not in a module or resource input. Below is some information about the error:

Error message
  * strconv.ParseInt: parsing "${length(split(\",\", var.public_subnets))}": invalid syntax
Variable output
$ terraform output public_subnets
public_subnets = 10.12.16.0/24,10.12.17.0/24,10.12.18.0/24
How the variable is generated from the public_subnets module
output "subnets" { value = "${join(",", aws_subnet.public.*.cidr_block)}" }
Then I pass it into my nat module
module "nat" {
  source            = "./nat"

  name              = "${var.name}-nat"
  vpc_id            = "${module.vpc_tools.vpc_id}"
  public_subnets    = "${module.admin_public_subnet.subnets}"
  subnet_ids        = "${module.admin_public_subnet.subnet_ids}"
}
The resource in the nat module
variable "public_subnets" { }

resource "aws_eip" "nat" {
  count = "${length(split(",", var.public_subnets))}"
  vpc   = true

  lifecycle {
    create_before_destroy = true
  }
}
When I pass the same CIDR list as a literal, the error disappears
module "nat" {
  source            = "./nat"

  name              = "${var.name}-nat"
  vpc_id            = "${module.vpc_tools.vpc_id}"
  public_subnets    = "10.12.16.0/24,10.12.17.0/24,10.12.18.0/24"
  subnet_ids        = "${module.admin_public_subnet.subnet_ids}"
}

colout commented Jan 14, 2016

I'm getting this error as well. It seems to have something to do with how the delimited list is generated. I can get this to work as an output variable, but not in a module or resource input. Below is some information about the error:

Error message
  * strconv.ParseInt: parsing "${length(split(\",\", var.public_subnets))}": invalid syntax
Variable output
$ terraform output public_subnets
public_subnets = 10.12.16.0/24,10.12.17.0/24,10.12.18.0/24
How the variable is generated from the public_subnets module
output "subnets" { value = "${join(",", aws_subnet.public.*.cidr_block)}" }
Then I pass it into my nat module
module "nat" {
  source            = "./nat"

  name              = "${var.name}-nat"
  vpc_id            = "${module.vpc_tools.vpc_id}"
  public_subnets    = "${module.admin_public_subnet.subnets}"
  subnet_ids        = "${module.admin_public_subnet.subnet_ids}"
}
The resource in the nat module
variable "public_subnets" { }

resource "aws_eip" "nat" {
  count = "${length(split(",", var.public_subnets))}"
  vpc   = true

  lifecycle {
    create_before_destroy = true
  }
}
When I pass the same CIDR list as a literal, the error disappears
module "nat" {
  source            = "./nat"

  name              = "${var.name}-nat"
  vpc_id            = "${module.vpc_tools.vpc_id}"
  public_subnets    = "10.12.16.0/24,10.12.17.0/24,10.12.18.0/24"
  subnet_ids        = "${module.admin_public_subnet.subnet_ids}"
}
@deftflux

This comment has been minimized.

Show comment
Hide comment
@deftflux

deftflux Jan 14, 2016

Contributor

So as I tried to explain in my previous comment, it has less to do with how complicated the interpolated string is, and more to do with whether or not its value can be computed at planning time.

"${var.count}" is as simple as it gets, but if the value of var.count depends on resources that haven't been created yet, then the value cannot be computed at the time when the count needs to be known. Apparently, the value of the count property needs to be resolved ahead of time so that Terraform can put together its resource graph and come up with a deployment plan.

That is why supplying a literal works fine. In that case, the value is not dependent on any resources having been created, so Terraform can compute the value ahead of time.

The error message "invalid syntax" is terribly misleading. It should at least say "value cannot be computed at this time" or something to that effect.

Contributor

deftflux commented Jan 14, 2016

So as I tried to explain in my previous comment, it has less to do with how complicated the interpolated string is, and more to do with whether or not its value can be computed at planning time.

"${var.count}" is as simple as it gets, but if the value of var.count depends on resources that haven't been created yet, then the value cannot be computed at the time when the count needs to be known. Apparently, the value of the count property needs to be resolved ahead of time so that Terraform can put together its resource graph and come up with a deployment plan.

That is why supplying a literal works fine. In that case, the value is not dependent on any resources having been created, so Terraform can compute the value ahead of time.

The error message "invalid syntax" is terribly misleading. It should at least say "value cannot be computed at this time" or something to that effect.

@justinclayton

This comment has been minimized.

Show comment
Hide comment
@justinclayton

justinclayton Jan 14, 2016

Contributor

That makes sense @deftflux . What still isn't making sense to me is why it's inconsistent. I will say that this happens much more often when running plan after a taint.

Contributor

justinclayton commented Jan 14, 2016

That makes sense @deftflux . What still isn't making sense to me is why it's inconsistent. I will say that this happens much more often when running plan after a taint.

@night0wl

This comment has been minimized.

Show comment
Hide comment
@night0wl

night0wl Jan 14, 2016

I am having the exact same issue as colout. It is also inconsistent, I have one VPC deployment where it works, but the same module placed into another VPC deployment throws this error.

In my use case I am defining var.public_subnets manually as "10.0.2.0/24,10.0.2.0/24". Works fine in one place, but not the other, using the same input method

night0wl commented Jan 14, 2016

I am having the exact same issue as colout. It is also inconsistent, I have one VPC deployment where it works, but the same module placed into another VPC deployment throws this error.

In my use case I am defining var.public_subnets manually as "10.0.2.0/24,10.0.2.0/24". Works fine in one place, but not the other, using the same input method

@colout

This comment has been minimized.

Show comment
Hide comment
@colout

colout Jan 14, 2016

@deftflux, what you said makes sense since the issue happens on simple interpolations as well as more complicated ones, but maybe you can help me understand the a few things I found in my testing:

  • In my case, the subnet resource was already deployed so it didn't have to be computed.
  • To rule out resource dependencies as the issue, I outputted subnet as a literal "10.12.16.0/24" from my admin_public_subnet module rather than computing it via "${join(",", aws_subnet.public.*.cidr_block)}". I get the same error message.
admin_public_subnet's output.tf
output "subnets" { value = "10.12.16.0/24" }
  • To see if this has anything to do with passing the output of a module, I changed my nat module input to the literal "10.12.16.0/24" rather than passing that same literal from "${module.admin_public_subnet.subnets}". The error went away.
from my root tf file
module "nat" {
  source              = "./nat"

  name                = "${var.name}-nat"
  vpc_id              = "${module.vpc_tools.vpc_id}"
  public_subnets      = "10.12.16.0/24"
  subnet_ids          = "${module.admin_public_subnet.subnet_ids}"
}

Maybe someone who has reproduced this in a simpler environment can confirm this, but things only seem to break for me when I pass a value (even a literal) from the output of another module. Is anyone seeing this who doesn't pass the variable via "${module.module_name.output_variable}"?

colout commented Jan 14, 2016

@deftflux, what you said makes sense since the issue happens on simple interpolations as well as more complicated ones, but maybe you can help me understand the a few things I found in my testing:

  • In my case, the subnet resource was already deployed so it didn't have to be computed.
  • To rule out resource dependencies as the issue, I outputted subnet as a literal "10.12.16.0/24" from my admin_public_subnet module rather than computing it via "${join(",", aws_subnet.public.*.cidr_block)}". I get the same error message.
admin_public_subnet's output.tf
output "subnets" { value = "10.12.16.0/24" }
  • To see if this has anything to do with passing the output of a module, I changed my nat module input to the literal "10.12.16.0/24" rather than passing that same literal from "${module.admin_public_subnet.subnets}". The error went away.
from my root tf file
module "nat" {
  source              = "./nat"

  name                = "${var.name}-nat"
  vpc_id              = "${module.vpc_tools.vpc_id}"
  public_subnets      = "10.12.16.0/24"
  subnet_ids          = "${module.admin_public_subnet.subnet_ids}"
}

Maybe someone who has reproduced this in a simpler environment can confirm this, but things only seem to break for me when I pass a value (even a literal) from the output of another module. Is anyone seeing this who doesn't pass the variable via "${module.module_name.output_variable}"?

@night0wl

This comment has been minimized.

Show comment
Hide comment
@night0wl

night0wl Jan 15, 2016

I am getting it passed through as a either a literal, or a top scope variable being passed into my module

module "vpc" {
  source = "./vpc"
  public_subnets = "10.0.1.0/24,10.0.2.0/24"
}

and

env.tfvars
public_subnets = "10.0.1.0/24,10.0.2.0/24"

modules.tf
module "vpc" {
  source = "./vpc"
  subnet_ids = "${var.public_subnets}"
}

yield the same result.

* Error reading aws_subnet.public_subnets count: strconv.ParseInt: parsing "${length(split(\",\", var.public_subnets))}": invalid syntax

This worked fine (and still works!) in one VPC built previously with it, but it is now failing with another

UPDATE:

After doing a bit more digging and using a git bisect to track this down, I found that I was looking in the wrong place; I use the exact line of code shown in the error message, however, I also used a splat to try and derive a count somewhere else in my code.

count = ${length(aws_subnet.public_subnets.*.id)}"

changing this resolved the error, so I now concur with the above statements, that this is to do with interpolation of computed resources and not literals.

night0wl commented Jan 15, 2016

I am getting it passed through as a either a literal, or a top scope variable being passed into my module

module "vpc" {
  source = "./vpc"
  public_subnets = "10.0.1.0/24,10.0.2.0/24"
}

and

env.tfvars
public_subnets = "10.0.1.0/24,10.0.2.0/24"

modules.tf
module "vpc" {
  source = "./vpc"
  subnet_ids = "${var.public_subnets}"
}

yield the same result.

* Error reading aws_subnet.public_subnets count: strconv.ParseInt: parsing "${length(split(\",\", var.public_subnets))}": invalid syntax

This worked fine (and still works!) in one VPC built previously with it, but it is now failing with another

UPDATE:

After doing a bit more digging and using a git bisect to track this down, I found that I was looking in the wrong place; I use the exact line of code shown in the error message, however, I also used a splat to try and derive a count somewhere else in my code.

count = ${length(aws_subnet.public_subnets.*.id)}"

changing this resolved the error, so I now concur with the above statements, that this is to do with interpolation of computed resources and not literals.

@colout

This comment has been minimized.

Show comment
Hide comment
@colout

colout Jan 15, 2016

I'm curious, does using the "${module.____.____}" behave similarly to computing a variable from a resource?

Again, I'm not computing the values in a resource. I'm outputting the value as a literal from a module then passing that to a second module where they are interpolated in count.

This fails
└── module "parent"
     ├── module "admin_public_subnet" { ... }
     │      └─ output "subnets" { value = "10.12.16.0/24,10.12.17.0/24" }
     │
     └── module "nat" { public_subnets = "${module.admin_public_subnet.subnets}" ... }
            └─ count = "${length(split(",", var.public_subnets))}"
 * strconv.ParseInt: parsing "${length(split(\",\", var.public_subnets))}": invalid syntax
This works
└── module "parent"
     └── module "nat" { public_subnets = "10.12.16.0/24,10.12.17.0/24" ... }
            └─ count = "${length(split(",", var.public_subnets))}"
public_subnets = 10.12.16.0/24,10.12.17.0/24

colout commented Jan 15, 2016

I'm curious, does using the "${module.____.____}" behave similarly to computing a variable from a resource?

Again, I'm not computing the values in a resource. I'm outputting the value as a literal from a module then passing that to a second module where they are interpolated in count.

This fails
└── module "parent"
     ├── module "admin_public_subnet" { ... }
     │      └─ output "subnets" { value = "10.12.16.0/24,10.12.17.0/24" }
     │
     └── module "nat" { public_subnets = "${module.admin_public_subnet.subnets}" ... }
            └─ count = "${length(split(",", var.public_subnets))}"
 * strconv.ParseInt: parsing "${length(split(\",\", var.public_subnets))}": invalid syntax
This works
└── module "parent"
     └── module "nat" { public_subnets = "10.12.16.0/24,10.12.17.0/24" ... }
            └─ count = "${length(split(",", var.public_subnets))}"
public_subnets = 10.12.16.0/24,10.12.17.0/24
@sc-chad

This comment has been minimized.

Show comment
Hide comment

sc-chad commented Jan 18, 2016

@jrnt30

This comment has been minimized.

Show comment
Hide comment
@jrnt30

jrnt30 Feb 11, 2016

Contributor

I am still experience the same issue as noted here #3884 (comment) with source code to reproduce the issue on 0.6.11

Contributor

jrnt30 commented Feb 11, 2016

I am still experience the same issue as noted here #3884 (comment) with source code to reproduce the issue on 0.6.11

@chadgrant

This comment has been minimized.

Show comment
Hide comment
@chadgrant

chadgrant Feb 21, 2016

same issue for me, subnets output from one module, input into another. all the count = "${length(split(",", var.public_subnet_ids))}" fail with parseInt errors.

chadgrant commented Feb 21, 2016

same issue for me, subnets output from one module, input into another. all the count = "${length(split(",", var.public_subnet_ids))}" fail with parseInt errors.

@mrwilby

This comment has been minimized.

Show comment
Hide comment
@mrwilby

mrwilby Feb 22, 2016

Same for me. Nothing new to add to the discussion as my situation is covered already in this thread.

mrwilby commented Feb 22, 2016

Same for me. Nothing new to add to the discussion as my situation is covered already in this thread.

@mikemackintosh

This comment has been minimized.

Show comment
Hide comment
@mikemackintosh

mikemackintosh Mar 7, 2016

Im using terraform_remote_state to populate the values passed to the module and experiencing the same behavior.

mikemackintosh commented Mar 7, 2016

Im using terraform_remote_state to populate the values passed to the module and experiencing the same behavior.

@colout

This comment has been minimized.

Show comment
Hide comment
@colout

colout Apr 19, 2016

All,

I found a thread that addresses the counting issue, and it appears a fix is in the works.

From #4169:
"This limitation on count exists because the value of count must be known during the plan phase, and thus it is impossible to complete the plan if the count value were a resource property that is not known until after apply."

The proposed solution in the linked thread is to create a new class of resource that is evaluated early ("First-class Data Sources"), and can be counted. It looks like a lot of work has already been done, and there are already a few pull requests out there.

I'd suggest reading through that thread and giving it a +1 if you like the solution.

colout commented Apr 19, 2016

All,

I found a thread that addresses the counting issue, and it appears a fix is in the works.

From #4169:
"This limitation on count exists because the value of count must be known during the plan phase, and thus it is impossible to complete the plan if the count value were a resource property that is not known until after apply."

The proposed solution in the linked thread is to create a new class of resource that is evaluated early ("First-class Data Sources"), and can be counted. It looks like a lot of work has already been done, and there are already a few pull requests out there.

I'd suggest reading through that thread and giving it a +1 if you like the solution.

@lmunro

This comment has been minimized.

Show comment
Hide comment
@lmunro

lmunro Aug 3, 2016

just tried v0.7.0 (which is great!) using the new data source resource in place of terraform_remote_state and this bug still seems to exist. Where var.aws_customer_gateways is an output from data source of type "terraform_remote_state"

This works, and the tag is populated fine:

resource "aws_s3_bucket" "test-bucket-can-delete" {
    bucket = "test-bucket-can-delete"
    acl = "private"

    tags {
        Test = "${length(var.aws_customer_gateways)}"
    }
}

This still throws a strconv.ParseInt invalid syntax error on the count attribute:

resource "aws_vpn_connection" "vpn_spoke" {
  count               = "${length(var.aws_customer_gateways)}"
  vpn_gateway_id      = "${aws_vpn_gateway.vpn_gateway.id}"
  customer_gateway_id = "${var.aws_customer_gateways[count.index]}"
  type                = "ipsec.1"
  static_routes_only  = false
}

lmunro commented Aug 3, 2016

just tried v0.7.0 (which is great!) using the new data source resource in place of terraform_remote_state and this bug still seems to exist. Where var.aws_customer_gateways is an output from data source of type "terraform_remote_state"

This works, and the tag is populated fine:

resource "aws_s3_bucket" "test-bucket-can-delete" {
    bucket = "test-bucket-can-delete"
    acl = "private"

    tags {
        Test = "${length(var.aws_customer_gateways)}"
    }
}

This still throws a strconv.ParseInt invalid syntax error on the count attribute:

resource "aws_vpn_connection" "vpn_spoke" {
  count               = "${length(var.aws_customer_gateways)}"
  vpn_gateway_id      = "${aws_vpn_gateway.vpn_gateway.id}"
  customer_gateway_id = "${var.aws_customer_gateways[count.index]}"
  type                = "ipsec.1"
  static_routes_only  = false
}
@skarungan

This comment has been minimized.

Show comment
Hide comment
@skarungan

skarungan Aug 7, 2016

Yes, same with 0.7.0 also, cannot use count = ${length(data.aws_availability_zones.available.names)}

skarungan commented Aug 7, 2016

Yes, same with 0.7.0 also, cannot use count = ${length(data.aws_availability_zones.available.names)}

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Aug 7, 2016

Also having the same problem when passing a variable into a module

the following doesn't work
peering_connection_id = "${data.terraform_remote_state.app.peering_connection_id}"

resource "aws_route" "peer_a" {
count = "${replace(replace(var.peering_connection_id,"/^$/",0),"/pcx-.*/","1")}"
route_table_id = "${aws_route_table.private_a.id}"
destination_cidr_block = "${var.destination_cidr_block}"
vpc_peering_connection_id = "${var.peering_connection_id}"
}

the following works:
peering_connection_id = "pcx-da9825b3"

resource "aws_route" "peer_a" {
count = "${replace(replace(var.peering_connection_id,"/^$/",0),"/pcx-.*/","1")}"
route_table_id = "${aws_route_table.private_a.id}"
destination_cidr_block = "${var.destination_cidr_block}"
vpc_peering_connection_id = "${var.peering_connection_id}"
}

Using Terraform v0.7.0

ghost commented Aug 7, 2016

Also having the same problem when passing a variable into a module

the following doesn't work
peering_connection_id = "${data.terraform_remote_state.app.peering_connection_id}"

resource "aws_route" "peer_a" {
count = "${replace(replace(var.peering_connection_id,"/^$/",0),"/pcx-.*/","1")}"
route_table_id = "${aws_route_table.private_a.id}"
destination_cidr_block = "${var.destination_cidr_block}"
vpc_peering_connection_id = "${var.peering_connection_id}"
}

the following works:
peering_connection_id = "pcx-da9825b3"

resource "aws_route" "peer_a" {
count = "${replace(replace(var.peering_connection_id,"/^$/",0),"/pcx-.*/","1")}"
route_table_id = "${aws_route_table.private_a.id}"
destination_cidr_block = "${var.destination_cidr_block}"
vpc_peering_connection_id = "${var.peering_connection_id}"
}

Using Terraform v0.7.0

@bfgoodrich

This comment has been minimized.

Show comment
Hide comment
@bfgoodrich

bfgoodrich Aug 9, 2016

+1 : This is seriously annoying, I can use an element interpolation for the same variable, but count doesn't work. Getting the same ParseInt error

Errors:

  * strconv.ParseInt: parsing "${length(split(\",\", var.vpn-peer-list))}": invalid syntax

bfgoodrich commented Aug 9, 2016

+1 : This is seriously annoying, I can use an element interpolation for the same variable, but count doesn't work. Getting the same ParseInt error

Errors:

  * strconv.ParseInt: parsing "${length(split(\",\", var.vpn-peer-list))}": invalid syntax
@brikis98

This comment has been minimized.

Show comment
Hide comment
@brikis98

brikis98 Aug 29, 2016

Contributor

Same issue here in Terraform 0.7.2. I wanted to take advantage of the new aws_availability_zones data source so I didn't have to specify the number of AZs manually, but due to this bug, it doesn't help much, as the following syntax does not work:

count = ${length(data.aws_availability_zones.available.names)}
Contributor

brikis98 commented Aug 29, 2016

Same issue here in Terraform 0.7.2. I wanted to take advantage of the new aws_availability_zones data source so I didn't have to specify the number of AZs manually, but due to this bug, it doesn't help much, as the following syntax does not work:

count = ${length(data.aws_availability_zones.available.names)}
@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh Aug 31, 2016

Member

AH HAH! I'm going to adopt this issue as THE ISSUE for this.

There appears to be two major issues here:

  1. Count that has a var across a module boundary
  2. Count that has a data source

What is not an issue (verified):

  1. Count that has a var in the root module (works).
  2. Count that has a non-datasource and non-variable reference (proper validation error).

I don't think all of this will be fixable all in one go but I think we can make this significantly better. Let me look into what can be done.

Member

mitchellh commented Aug 31, 2016

AH HAH! I'm going to adopt this issue as THE ISSUE for this.

There appears to be two major issues here:

  1. Count that has a var across a module boundary
  2. Count that has a data source

What is not an issue (verified):

  1. Count that has a var in the root module (works).
  2. Count that has a non-datasource and non-variable reference (proper validation error).

I don't think all of this will be fixable all in one go but I think we can make this significantly better. Let me look into what can be done.

@brikis98

This comment has been minimized.

Show comment
Hide comment
@brikis98

brikis98 Sep 1, 2016

Contributor

@mitchellh Is the fix going to be to just report better error messages or to actually support dynamic data in count, such as count = ${length(data.aws_availability_zones.available.names)}?

Contributor

brikis98 commented Sep 1, 2016

@mitchellh Is the fix going to be to just report better error messages or to actually support dynamic data in count, such as count = ${length(data.aws_availability_zones.available.names)}?

@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh Sep 1, 2016

Member

@brikis98 At a minimum certainly that, but hopefully I can do better. The intent for count was always to just have a good UX on restrictions early on, and expand it over time. The error that many many people are getting shown in this thread is pretty bad. At the least we need to fix that.

Member

mitchellh commented Sep 1, 2016

@brikis98 At a minimum certainly that, but hopefully I can do better. The intent for count was always to just have a good UX on restrictions early on, and expand it over time. The error that many many people are getting shown in this thread is pretty bad. At the least we need to fix that.

@rtlong

This comment has been minimized.

Show comment
Hide comment

rtlong commented Sep 11, 2016

@pmoust

This comment has been minimized.

Show comment
Hide comment
@pmoust

pmoust Oct 14, 2016

Contributor

I would love to see this one getting a priority boost.
It is a blocker in adopting nested modules in complex (and in extend even trivial) setups.

Our use case would benefit a lot from being able to use interpolation functions in modules to deduct count based on passed variables or data sources.
Basically what is described in #3888 (comment) stands 100% and is the blocker - so I will refrain from sharing extra repro examples, they are not much different to what others have provided in this thread already.
I'll just mention that still stands in 0.7.5

I tried to give it a go and tackling it aiming for a clean PR, but it turned out to be much more time consuming for me than I would like, to the point I simply gave up.

We are about to design our module hierarchy for a cross-region cross-provider setup, so I'd like to know what's the priority status on this issue.

If it's something that won't be landing in next month's release (if there's one to be rolled out), we might choose to flatten our TF approach and hardcode stuff to bypass this issue.

Contributor

pmoust commented Oct 14, 2016

I would love to see this one getting a priority boost.
It is a blocker in adopting nested modules in complex (and in extend even trivial) setups.

Our use case would benefit a lot from being able to use interpolation functions in modules to deduct count based on passed variables or data sources.
Basically what is described in #3888 (comment) stands 100% and is the blocker - so I will refrain from sharing extra repro examples, they are not much different to what others have provided in this thread already.
I'll just mention that still stands in 0.7.5

I tried to give it a go and tackling it aiming for a clean PR, but it turned out to be much more time consuming for me than I would like, to the point I simply gave up.

We are about to design our module hierarchy for a cross-region cross-provider setup, so I'd like to know what's the priority status on this issue.

If it's something that won't be landing in next month's release (if there's one to be rolled out), we might choose to flatten our TF approach and hardcode stuff to bypass this issue.

@lordnynex

This comment has been minimized.

Show comment
Hide comment
@lordnynex

lordnynex Oct 18, 2016

When will these bugs be fixed? I'm trying to be as patient as possible with terraform but I've been blocked for months now on issues like this. I'm growing impatient because I have hashicorp reps pressing me to sign several hundred thousand dollar enterprise contracts, yet somehow simple count bugs are going overlooked for a year at a time.

For a project this mature that costs this much money I expect

  • Things like count looping bugs to be addressed with the level of urgency they deserve.
  • The hashicorp issue trackers need to contain meaningful information about the issue and the projected resolution. A year old thread of 'alternative workarounds' are not ok. The nature of this bug is at direct odds with the products documentation.
  • The hashicorp responses need to choose language that conveys they take these issues seriously. Using words like 'complex' to describe spreading resources over multiple availability zones is a bit insulting. This is an absolutely standard practice, and is required by providers for SLAs. The product does not make sense if it comes at the expense of industry standards.

lordnynex commented Oct 18, 2016

When will these bugs be fixed? I'm trying to be as patient as possible with terraform but I've been blocked for months now on issues like this. I'm growing impatient because I have hashicorp reps pressing me to sign several hundred thousand dollar enterprise contracts, yet somehow simple count bugs are going overlooked for a year at a time.

For a project this mature that costs this much money I expect

  • Things like count looping bugs to be addressed with the level of urgency they deserve.
  • The hashicorp issue trackers need to contain meaningful information about the issue and the projected resolution. A year old thread of 'alternative workarounds' are not ok. The nature of this bug is at direct odds with the products documentation.
  • The hashicorp responses need to choose language that conveys they take these issues seriously. Using words like 'complex' to describe spreading resources over multiple availability zones is a bit insulting. This is an absolutely standard practice, and is required by providers for SLAs. The product does not make sense if it comes at the expense of industry standards.
@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh Oct 27, 2016

Member

@lordnynex I'm happy to respond to this.

This bug is not simple, unfortunately. Reproducing it is very, very simple. The fix is far from it. We've recently started a big refactor to make our implementation of various operations more robust (and simpler per operation, more complicated from the outside), which we believe will enable much easier fixes for things like this. Unfortunately as you might expect its a fairly large undertaking.

We recently merged a couple of this massive changes in PRs such as #9388. That lays a lot of the groundwork needed to be able to fix a bug like this. I still can't give you a timeline for when it will be fixed because it matters how much bandwidth I can find. I'm being pushed to get a Terraform 0.8 release out with the major features (and hundreds of bug fixes) that we already have. I want to find time to work on the refactor necessary to fix this but I'm not quite there yet.

To give a solid timeline: probably in a few months, but not before.

Of course, this is an issue in the open source aspect of the project, so if anyone is willing to take a look (it won't be easy but its definitely possible) then we'd be infinitely grateful. We're burning down issues very quickly though! (See https://github.com/hashicorp/terraform/pulse/monthly)

Sorry, not the best news. But I'm going to underpromise here and try to deliver sooner.

Member

mitchellh commented Oct 27, 2016

@lordnynex I'm happy to respond to this.

This bug is not simple, unfortunately. Reproducing it is very, very simple. The fix is far from it. We've recently started a big refactor to make our implementation of various operations more robust (and simpler per operation, more complicated from the outside), which we believe will enable much easier fixes for things like this. Unfortunately as you might expect its a fairly large undertaking.

We recently merged a couple of this massive changes in PRs such as #9388. That lays a lot of the groundwork needed to be able to fix a bug like this. I still can't give you a timeline for when it will be fixed because it matters how much bandwidth I can find. I'm being pushed to get a Terraform 0.8 release out with the major features (and hundreds of bug fixes) that we already have. I want to find time to work on the refactor necessary to fix this but I'm not quite there yet.

To give a solid timeline: probably in a few months, but not before.

Of course, this is an issue in the open source aspect of the project, so if anyone is willing to take a look (it won't be easy but its definitely possible) then we'd be infinitely grateful. We're burning down issues very quickly though! (See https://github.com/hashicorp/terraform/pulse/monthly)

Sorry, not the best news. But I'm going to underpromise here and try to deliver sooner.

@dmatthis-sm

This comment has been minimized.

Show comment
Hide comment
@dmatthis-sm

dmatthis-sm Nov 15, 2016

Is it possible to force an order of operations option? If I make a module that has my main infrastructure items and I need it to generate it along with its subnets and gatweays before I build instances then my being able to have my VPC module build first along would resolve most issues with nodes missing the ability to find their resource. Then the data from the output for modules that were forced to generate first could be made available to the VMs.

dmatthis-sm commented Nov 15, 2016

Is it possible to force an order of operations option? If I make a module that has my main infrastructure items and I need it to generate it along with its subnets and gatweays before I build instances then my being able to have my VPC module build first along would resolve most issues with nodes missing the ability to find their resource. Then the data from the output for modules that were forced to generate first could be made available to the VMs.

@derekstavis

This comment has been minimized.

Show comment
Hide comment
@derekstavis

derekstavis Nov 29, 2016

@mitchellh Thanks for providing an insider look on how difficult this issue is.

I have noted that debugging the computation of length works when it's used on a resource tag, for example, and this brought us to do a quick pair programming here at Pagar.me, trying some approaches to fix this issue in TF code, but we were unsuccessful.

Could you give any advice on how to continue digging to achieve a fix for this issue? We have been looking to work on a fix for it and would like to hear from an insider to ease the process.

derekstavis commented Nov 29, 2016

@mitchellh Thanks for providing an insider look on how difficult this issue is.

I have noted that debugging the computation of length works when it's used on a resource tag, for example, and this brought us to do a quick pair programming here at Pagar.me, trying some approaches to fix this issue in TF code, but we were unsuccessful.

Could you give any advice on how to continue digging to achieve a fix for this issue? We have been looking to work on a fix for it and would like to hear from an insider to ease the process.

greenboxal added a commit to greenboxal/terraform that referenced this issue Nov 29, 2016

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.
@derekstavis

This comment has been minimized.

Show comment
Hide comment
@derekstavis

derekstavis Nov 29, 2016

Hey @mitchellh! We have achieved a solution for count expansion using a hackish way in #10418. It have proved to be working on our minimal example. Refer to the pull request for further discussion.

derekstavis commented Nov 29, 2016

Hey @mitchellh! We have achieved a solution for count expansion using a hackish way in #10418. It have proved to be working on our minimal example. Refer to the pull request for further discussion.

greenboxal added a commit to greenboxal/terraform that referenced this issue Nov 29, 2016

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.

greenboxal added a commit to greenboxal/terraform that referenced this issue Dec 15, 2016

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.
@tomgoren

This comment has been minimized.

Show comment
Hide comment
@tomgoren

tomgoren Dec 16, 2016

Contributor

Pretty sure this is also a dupe of #1497

Contributor

tomgoren commented Dec 16, 2016

Pretty sure this is also a dupe of #1497

greenboxal added a commit to greenboxal/terraform that referenced this issue Jan 24, 2017

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.

mitchellh added a commit that referenced this issue Jan 28, 2017

terraform: don't validate computed values in validate
This disables the computed value check for `count` during the validation
pass. This enables partial support for #3888 or #1497: as long as the
value is non-computed during the plan, complex values will work in
counts.

**Notably, this allows data source values to be present in counts!**

The "count" value can be disabled during validation safely because we
can treat it as if any field that uses `count.index` is computed for
validation. We then validate a single instance (as if `count = 1`) just
to make sure all required fields are set.
@mitchellh

This comment has been minimized.

Show comment
Hide comment
@mitchellh

mitchellh Jan 29, 2017

Member

I agree @tomgoren. It is confusing where to post since both are very valid.

This is nearly fixed now with #11482! I'm going to close this as a dup of #1497 now, and will close #1497 when #11482 is merged. See that for an update.

Member

mitchellh commented Jan 29, 2017

I agree @tomgoren. It is confusing where to post since both are very valid.

This is nearly fixed now with #11482! I'm going to close this as a dup of #1497 now, and will close #1497 when #11482 is merged. See that for an update.

@mitchellh mitchellh closed this Jan 29, 2017

@tomgoren

This comment has been minimized.

Show comment
Hide comment
@tomgoren

tomgoren Jan 29, 2017

Contributor

Thank you @mitchellh!

Contributor

tomgoren commented Jan 29, 2017

Thank you @mitchellh!

greenboxal added a commit to greenboxal/terraform that referenced this issue Feb 16, 2017

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.

greenboxal added a commit to greenboxal/terraform that referenced this issue Feb 18, 2017

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.

nusnewob added a commit to uktrade/terraform-module-aws-vpc that referenced this issue Mar 16, 2017

DennyLoko added a commit to DennyLoko/terraform that referenced this issue Jun 6, 2017

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.

greenboxal added a commit to greenboxal/terraform that referenced this issue Sep 1, 2017

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.

DennyLoko added a commit to pagarme/terraform that referenced this issue Dec 20, 2017

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.

wilkmaia added a commit to pagarme/terraform that referenced this issue Jan 16, 2018

Fix count interpolation with computed values
This commit enables the use computed values when interpolating
the count special variable.

This is achieved by defering the expansion of `count` during
walkInput and walkValidate as the real value isn't necessary
at this point.

The value will be resolved during walkRefresh and DynamicExpand
will work as expected.

Partially fixes #3888.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment