error using variables exported from one module in another

[teancom]

The problem

I have two terraform modules - one to create a VPC, a couple subnets and a nat server, the other to create three more subnets and an elastic IP (this is all in the service of creating a flexible system for installing Cloud Foundry). However, when I try and use the two modules together, using some outputs from the first as variables in the second (VPC id, route tables, availability zones), it fails. Oddly enough, it fails complaining about a different variable each time, like there's a race condition or a hash that's in a different order.

Error output

$ make plan
terraform get -update
Get: git::https://github.com/cloudfoundry-community/terraform-aws-vpc.git (update)
Get: git::https://github.com/cloudfoundry-community/terraform-aws-cf.git (update)
terraform plan -module-depth=-1 -var-file terraform.tfvars -out terraform.tfplan
Refreshing Terraform state prior to plan...

Error refreshing state: Module 'vpc' not found for variable 'module.vpc.aws_route_table_private_id'
make: *** [plan] Error 1

However, like I said, the variable in question changes almost every time. Running that again, I get:
Error refreshing state: Module 'vpc' not found for variable 'module.vpc.aws_route_table_public_id'

Then I get that two more times in a row, then:
Error refreshing state: Module 'vpc' not found for variable 'module.vpc.aws_internet_gateway_id'

To reproduce

1. git clone https://github.com/teancom/terraform-test
2. cp terraform.tfvars-example terraform.tfvars
3. Fill terraform.tfvars in with valid values
4. make plan

Workaround / Possible Clue

The modules work if you comment out the cf module completely, run make plan and then make apply, wait for it to successfully complete, and then uncomment the cf module and run terraform again.

[teancom]

Sorry, forgot to mention this is with 0.3.5 on OS X 10.10.2

[lnguyen]

Looks like this is the commit that broke things e82a285 . Seem like error checking is causing it have unwanted errors.

[teancom]

I can confirm that rolling back to ca48dca works.

[armon]

Thanks for the repro case, hopefully we can dive in soon. The commit in question actually just adds error checking for some of the internal methods. Previously, we weren't checking for certain interpolation errors which caused other crashes / panics. The fact that errors are now cropping up means we were masking bugs previously.

[teancom]

Long and I were discussing that. I was - without actually looking at or understanding the code - propositioning that it was possible the error checking itself was the cause of the problem, if it was looking for a variable that wouldn't be there until it was applying (i.e, a variable derived from another variable that is marked as 'calculated'). But you know your code better than I do (obviously).

Let me know if there is anything I can do to help track this down.

[kyriakosoikonomakos]

I hit a this issue too. Running "terraform apply -refresh=false" worked around the problem, but this may not always be the right thing to do.

[mitchellh]

Reproduced, super good repro steps, thanks!

[mitchellh]

I've got a fix... I just can't get a reproducing unit test, which is blocking the fix right now. Ugh! Working on that.

[mitchellh]

Fixed!

[lnguyen]

Thanks!!

[teancom]

Yes, thank you very much!

[teancom]

@mitchellh Hrrm. I'm running HEAD with the fix, and I don't get the same error I got before (i.e., make plan works great), but I do get the following when running make apply:

Error applying plan:

aws_security_group.cf: diffs didn't match during apply. This is a bug with the resource provider, please report a bug.

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

I've tried it a few times and get the same problem each time. Can you run through the repro steps I listed before and see if you are able to successfully make apply?

[teancom]

I should say, when running ca48dca not only did make plan work, but make apply worked as well. This particular diffs didn't match during apply bug didn't crop up until now.

[teancom]

If you think this is a problem with something else, unrelated to the changes you made, let me know and I'll open a different ticket.

[mitchellh]

I'll reopen to look at this.

The issue with the commit passed ca48dca was it was failing, we just weren't surfacing the error. The fact it happened to work down the road was purely coincidental and was just covering bugs. We're fixing this properly now (with unit tests) and will get to a stable point for this.

[mitchellh]

@teancom I think this might be fixed? I just ran from master and got a successful apply. Can you try again? I'll also try again to see if maybe this is a race of some sort...

[teancom]

Man, what the hell. I just was successful twice in a row too. Now, admittedly there are a few commits between ef60ac5 and 8347569 (I was running ef60ac5), but they all look like tests, not code changes.

[mitchellh]

Maybe you ran Terraform just before the compile finished? :) I dunno. I found a couple other bugs I'm fixing now related to this.

[teancom]

No, the obvious correct answer is that this bug only affects people in the mid-afternoon, EST, when they haven't had their nap.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.