Updating instance profile used by elasticbeanstalk environment breaks

[oillio]

I am not sure this is in scope for Terraform or not.
I have an aws_iam_instance_profile that is referenced by an aws_elastic_beanstalk_environment.
If the profile is modified, Terraform may need to delete and re-create the profile (in my case, the name changed).

Terraform deletes and re-creates the profile before updating the beanstalk environment to point to the new profile. It appears that, when the original profile is deleted, elastic beanstalk gets into an invalid state and has some very odd behavior.

Ideally, Terraform would create the new profile, update the elastic beanstalk environment, and only then delete the old profile.

[catsby]

Hey @oillio – do you have an sample configuration that demonstrates how you're connecting the IAM instance profile and your Beanstalk env? That will help me dig into this.

Thanks!

[oillio]

This is a simplified example which shows the link. If it isn't obvious from this, I can write up a complete test example.

If this applied, change old_profile_name. Terraform should delete and re-create the profile.

resource "aws_iam_instance_profile" "subject_profile" {
  name = "old_profile_name"
  roles = ["${aws_iam_role.some_role.name}"]
}

resource "aws_elastic_beanstalk_environment" "myEnv" {
  name = "myEnv"
  application = "myApp"
  description = "An example environment"

  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name = "IamInstanceProfile"
    value = "${aws_iam_instance_profile.subject_profile.name}"
  }
}

[bobbydeveaux]

Having a similar issue but my instance profile is hardcoded. It's never changing. However, each time I run terraform apply, it changes my IamInstanceProfile:

Apply complete! Resources: 0 added, 4 changed, 0 destroyed.

setting.2935929242.name:      "" => "IamInstanceProfile"
setting.2935929242.namespace: "" => "aws:autoscaling:launchconfiguration"
setting.2935929242.value:     "" => "arn:aws:iam::345345345345:instance-profile/eb-iam-profile"

setting.3177677351.name:      "IamInstanceProfile" => ""
setting.3177677351.namespace: "aws:autoscaling:launchconfiguration" => ""
setting.3177677351.value:     "eb-iam-profile" => ""

[dharrisio]

@bobbydeveaux in that case, I would recommend just using the profile name for now. The Elastic Beanstalk API doesn't store the whole arn, so when the environment is read there will always be a diff if your Terraform document contains the arn.

There are a few other normalization issues similar to this, so I might create a separate issue to track those. Unfortunately, there isn't really a good general solution for this, and it will have to be fixed on a case by case basis.

[dharrisio]

@oillio Would using create_before_destroy on the profile work in this case? If not I'm not really sure how to get around that issue, other than creating a new profile resource and updating the link to use that new profile. After applying that change, you could then delete the old profile, using -target and remove the old profile from your Terraform document. Not an ideal solution, but I think it would work.

[bobbydeveaux]

Ah - perfect, thanks @dharrisio :)

[oillio]

Yep. I can find a workaround to make it work.
The primary reason I submitted the issue is because it was not obvious to me that I was doing something dangerous when I originally did this, and the result was pretty extreme and a pain to fix.

I would rather an error message than what happened. If possible, I'd just like to avoid anyone else going down the path I did.

[arsdehnel]

I think I'm having an issue related to this although my problem it's causing is slightly different. In my configuration I'm specifying the Instance Profile ARN as the Terraform docs and AWS docs seem to indicate is acceptable. That is working fine to associate the profile with the instances (yay!) but I am having the same issue described above where every time I run plan or apply Terraform is recreating the environment due to the mismatch of instance profile name (stored in the state files) and instance profile arn (provided in my config).

To resolve this I went with the instance profile name in my configuration which did indeed stop the constant rebuilds. But I just noticed it also stopped being applied correctly in elastic beanstalk. My instance now are coming up with no instance profile associated with them which is noticeably worse than the constant rebuilds.

[arsdehnel]

This definitely appears to be a bug in Terraform and it's a little annoying. Not breaking anything but it's frustrating nonetheless. Right now I'm using a config like this:

resource "aws_elastic_beanstalk_environment" "eb_env" {
 
    name = "my_env_name"
    application = "my_app_name"
    solution_stack_name = "64bit Amazon Linux 2016.09 v3.3.0 running Node.js"
     
    setting {
        namespace = "aws:autoscaling:launchconfiguration"
        name      = "IamInstanceProfile"
        value     = "arn:aws:iam::1234567890:instance-profile/beanstalk_instance_profile"
    }
 
}

And that ends up with terraform apply changing the resource every single time. If I change the config to this where I just use the profile name instead of the full arn:

resource "aws_elastic_beanstalk_environment" "eb_env" {
 
    name = "my_env_name"
    application = "my_app_name"
    solution_stack_name = "64bit Amazon Linux 2016.09 v3.3.0 running Node.js"
     
    setting {
        namespace = "aws:autoscaling:launchconfiguration"
        name      = "IamInstanceProfile"
        value     = "beanstalk_instance_profile"
    }
 
}

In this instance terraform apply stops recreating the resource but it also doesn't apply the instance profile properly for me (the AWS console shows it as no profile associated).

[grebois]

+1, also hitting @arsdehnel issue under a very similar configuration:

Error applying plan:

1 error(s) occurred:

* aws_elastic_beanstalk_environment.environment:
2017-02-22 10:22:15.61 +0000 UTC: Failed to deploy configuration.
2017-02-22 10:22:15.399 +0000 UTC: Environment must have instance profile associated with it.

[Techbrunch]

I confirm that the same problem is happening for us. We are passing the arn but a change is detected everytime. Terraform v0.8.8

According to the AWS documentation:

An instance profile enables AWS Identity and Access Management (IAM) users and AWS services to access temporary security credentials to make AWS API calls. Specify the profile name or the ARN.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.