Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't deploy MySQL 5.7 Aurora clusters InvalidParameterCombination: Cannot find version 5.7.mysql_aurora.2.03.2 for aurora #10585

Open
bhechinger opened this issue Oct 21, 2019 · 10 comments
Labels
bug service/ec2 service/rds

Comments

@bhechinger
Copy link

@bhechinger bhechinger commented Oct 21, 2019

Community Note

  • Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.12
+ provider.aws v2.33.0

Affected Resource(s)

  • aws_rds_cluster_instance

Terraform Configuration Files

provider "aws" {
  version = "~> 2.0"
  region = "us-east-1"
}

resource "aws_vpc" "main" {
  cidr_block = "10.0.0.0/16"
}

resource "aws_subnet" "subnet-1a" {
  vpc_id = aws_vpc.main.id
  cidr_block = "10.0.1.0/24"
  availability_zone = "us-east-1a"

  tags = {
    Name = "subnet-1a"
  }
}

resource "aws_subnet" "subnet-1b" {
  vpc_id = aws_vpc.main.id
  cidr_block = "10.0.2.0/24"
  availability_zone = "us-east-1b"

  tags = {
    Name = "subnet-1b"
  }
}

resource "aws_subnet" "subnet-1c" {
  vpc_id = aws_vpc.main.id
  cidr_block = "10.0.3.0/24"
  availability_zone = "us-east-1c"

  tags = {
    Name = "subnet-1c"
  }
}

resource "aws_db_subnet_group" "aurora_subnet_group" {
  name = "aurora_db_subnet_group"
  description = "Allowed subnets for Aurora DB cluster instances"
  subnet_ids = [
    aws_subnet.subnet-1a.id,
    aws_subnet.subnet-1b.id,
    aws_subnet.subnet-1c.id
  ]
}

resource "aws_rds_cluster" "default" {
  cluster_identifier = "aurora-cluster-demo"
  engine = "aurora-mysql"
  engine_version = "5.7.mysql_aurora.2.03.2"
  availability_zones = [
    "us-east-1a",
    "us-east-1b",
    "us-east-1c"]
  database_name = "mpos"
  master_username = "root"
  master_password = "SUPERSEKRITPASSWORD"
  backup_retention_period = 5
  preferred_backup_window = "07:00-09:00"
  db_subnet_group_name = aws_db_subnet_group.aurora_subnet_group.name
}

resource "aws_rds_cluster_instance" "aurora_cluster_instance" {
  count = 3

  identifier = "aurora-instance-${count.index}"
  cluster_identifier = aws_rds_cluster.default.id
  instance_class = "db.t2.small"
  db_subnet_group_name = aws_db_subnet_group.aurora_subnet_group.name
  publicly_accessible = false

  lifecycle {
    create_before_destroy = true
  }
}

Debug Output

https://gist.github.com/bhechinger/270be3f8d28fb0e37ec72af739164762

Expected Behavior

Terraform should have created the RDS instances

Actual Behavior

Terraform didn't create the RDS instances

Steps to Reproduce

  1. terraform apply

Important Factoids

This is a clean account with nothing else in it. I had this issue in the past with 0.10 or 0.11 but I no longer remember the details. I'm very surprised this is still happening as that was well over a year ago.

This isn't some fancy example, either. This is pulled straight out of the terraform documentation and yet doesn't work.

I tried bumping the version up to 5.7.mysql_aurora.2.04.6 but that has the same exact behavior.

@ghost ghost added service/ec2 service/rds labels Oct 21, 2019
@github-actions github-actions bot added the needs-triage label Oct 21, 2019
@bhechinger
Copy link
Author

@bhechinger bhechinger commented Oct 22, 2019

So what it looks like is somehow it's setting the engine to aurora when it makes the call to AWS instead of setting it to aurora-mysql which angers AWS as that's not a valid combination.

Digging through the code now to try and figure out why but I have never looked at a terraform provider before so this is slow going. :)

2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4: 2019/10/21 15:31:57 [DEBUG] Creating RDS DB Instance opts: {
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   AutoMinorVersionUpgrade: true,
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   CopyTagsToSnapshot: false,
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   DBClusterIdentifier: "aurora-cluster-demo",
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   DBInstanceClass: "db.t2.small",
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   DBInstanceIdentifier: "aurora-instance-2",
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   DBSubnetGroupName: "aurora_db_subnet_group",
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   Engine: "aurora",
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   PromotionTier: 0,
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   PubliclyAccessible: false,
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4:   Tags: []
2019-10-21T15:31:57.379-0400 [DEBUG] plugin.terraform-provider-aws_v2.33.0_x4: }

@bhechinger
Copy link
Author

@bhechinger bhechinger commented Oct 22, 2019

Ok, so adding:

  engine = "aurora-mysql"
  engine_version = "5.7.mysql_aurora.2.03.2"

to the aws_rds_cluster_instance block as well works. I don't know, however, if this is correct behavior and a documentation error or if the instance should be getting the engine/version from the cluster.

@chenbr
Copy link

@chenbr chenbr commented Nov 1, 2019

Bump..

Ok, so adding:

  engine = "aurora-mysql"
  engine_version = "5.7.mysql_aurora.2.03.2"

to the aws_rds_cluster_instance block as well works. I don't know, however, if this is correct behavior and a documentation error or if the instance should be getting the engine/version from the cluster.

Bump.. just hit this as well.. any timeline for a fix?

@alexwlchan
Copy link
Contributor

@alexwlchan alexwlchan commented Nov 26, 2019

I think this is the code where Terraform reads the engine/engine version, which is done entirely on the basis of the supplied attributes, not the aws_rds_cluster resource:

https://github.com/terraform-providers/terraform-provider-aws/blob/acc6a21a3bd6159f4f8c10a86bcd08e70edce4f2/aws/resource_aws_rds_cluster_instance.go#L218-L227

You do have access to an RDS client (conn := meta.(*AWSClient).rdsconn), but trying to work out the correct engine type/version could be fiddly.

  • What if the user supplies an engine parameter that doesn't match the RDS cluster?
  • What if the user creates an aws_rds_cluster without an engine parameter, then aws_rds_cluster_instance that sets a non-default engine parameter?
  • What if the user is creating the aws_rds_cluster at the same time as the instances, so you have to go digging through Terraform state to find out what engine the about-to-be-created cluster will use?

Not to say it can't be done, but it's probably not an easy patch.

I鈥檝e just run into this problem, possibly from working the same example that @bhechinger was hitting (https://www.terraform.io/docs/providers/aws/r/rds_cluster_instance.html). As a short-term fix, what about making this small change to the example:

 resource "aws_rds_cluster_instance" "cluster_instances" {
   count              = 2
   identifier         = "aurora-cluster-demo-${count.index}"
   cluster_identifier = "${aws_rds_cluster.default.id}"
   instance_class     = "db.r4.large"
+  engine             = "${aws_rds_cluster.engine}"
+  engine_version     = "${aws_rds_cluster.engine_version}"
 }

 resource "aws_rds_cluster" "default" {
   cluster_identifier = "aurora-cluster-demo"
   availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
   database_name      = "mydb"
   master_username    = "foo"
   master_password    = "barbut8chars"
 }

It highlights that the two need to match, and anybody who copies and then adapts the example is going to get the correct behaviour. This would probably have helped me avoid making the same mistake, or at least caught it sooner.

What do other people think?

@eduardopuente
Copy link

@eduardopuente eduardopuente commented Jul 2, 2020

same problem here, doing what @alexwlchan says works. Would be great to have this working without the engine and engine_version in the aws_rds_cluster_instance resource

bflad pushed a commit that referenced this issue Jul 6, 2020
鈥 cluster (#14037)

If you try to create an RDS instance with a different engine/version to
the cluster, you get an InvalidParameterCombination error from Terraform
and no instances.

Changing the behaviour so that instances inherit their config from the
cluster is a big change.

This patch just tweaks the documented examples, so if people copy/paste
the example and change the engine/version of the cluster, it will still
work correctly.  It also highlights that the engine/version should
match between the instance and the cluster.

Relates to #10585
@justinretzolk
Copy link
Member

@justinretzolk justinretzolk commented Oct 14, 2021

Hey @bhechinger 馃憢 Thank you for taking the time to file this issue. Given that there's been a number of AWS provider releases since you initially filed it, can you confirm whether you're still experiencing this issue?

@justinretzolk justinretzolk added waiting-response and removed needs-triage labels Oct 14, 2021
@bhechinger
Copy link
Author

@bhechinger bhechinger commented Oct 14, 2021

Wow, it's been a while since I was even doing this! 馃ぃ

I will do some testing for you, however. It's the least I could do.

@github-actions github-actions bot removed the waiting-response label Oct 14, 2021
@justinretzolk
Copy link
Member

@justinretzolk justinretzolk commented Oct 14, 2021

@bhechinger We appreciate it a ton! Apologies for the delay in getting back to you; we're doing quite a lot of work in trying to better that experience (including my being in this dedicated role 馃槂). I'll keep an eye out for any updates you may have.

@justinretzolk justinretzolk added the waiting-response label Oct 14, 2021
@edlinklater
Copy link

@edlinklater edlinklater commented Dec 15, 2021

Just confirming that I've run into the same issue just now, so it still seems to be an issue running latest 3.x.

@github-actions github-actions bot removed the waiting-response label Dec 15, 2021
@justinretzolk justinretzolk added the bug label Dec 16, 2021
@be4ndr
Copy link

@be4ndr be4ndr commented Jan 6, 2022

I found a root cause.
Amazon changed an engine name for Aurora MySQL 2.x
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Updates.20180206.html#AuroraMySQL.Updates.20180206.CLI
The engine name for Aurora MySQL 2.x is aurora-mysql; the engine name for Aurora MySQL 1.x continues to be aurora.

So in my case when I used these variables terraform successfully built Aurora RDS.
aurora_engine = "aurora-mysql"
aurora_engine_version = "5.7.mysql_aurora.2.10.1"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug service/ec2 service/rds
Projects
None yet
Development

No branches or pull requests

7 participants