Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on creating a ManagedKubernetes Cluster #1034

Closed
paolomainardi opened this issue Apr 18, 2019 · 11 comments

Comments

@paolomainardi
Copy link

@paolomainardi paolomainardi commented Apr 18, 2019

Terraform Version

0.11.7

Affected Resource(s)

  • alicloud_cs_managed_kubernetes

Terraform Configuration Files

resource "alicloud_vswitch" "vswitches" {
  count             = "${length(var.vswitch_ids) > 0 ? 0 : length(var.vswitch_cidrs)}"
  vpc_id            = "${var.vpc_id == "" ? join("", alicloud_vpc.vpc.*.id) : var.vpc_id}"
  cidr_block        = "${element(var.vswitch_cidrs, count.index)}"
  availability_zone = "${lookup(data.alicloud_zones.main.zones[count.index%length(data.alicloud_zones.main.zones)], "id")}"
  name              = "${var.vswitch_name_prefix == "" ? format("%s-%s", var.example_name, format(var.number_format, count.index+1)) : format("%s-%s", var.vswitch_name_prefix, format(var.number_format, count.index+1))}"
}

resource "alicloud_nat_gateway" "default" {
  count  = "${var.new_nat_gateway == true ? 1 : 0}"
  vpc_id = "${var.vpc_id == "" ? join("", alicloud_vpc.vpc.*.id) : var.vpc_id}"
  name   = "${var.example_name}"
  specification = "Small"
}

resource "alicloud_eip" "default" {
  count     = "${var.new_nat_gateway == "true" ? 1 : 0}"
  bandwidth = "${var.eip_bandwidth}"
}

resource "alicloud_eip_association" "default" {
  count         = "${var.new_nat_gateway == "true" ? 1 : 0}"
  allocation_id = "${alicloud_eip.default.id}"
  instance_id   = "${alicloud_nat_gateway.default.id}"
}

resource "alicloud_snat_entry" "default" {
  count             = "${var.new_nat_gateway == "false" ? 0 : length(var.vswitch_ids) > 0 ? length(var.vswitch_ids) : length(var.vswitch_cidrs)}"
  snat_table_id     = "${alicloud_nat_gateway.default.snat_table_ids}"
  source_vswitch_id = "${length(var.vswitch_ids) > 0 ? element(split(",", join(",", var.vswitch_ids)), count.index%length(split(",", join(",", var.vswitch_ids)))) : length(var.vswitch_cidrs) < 1 ? "" : element(split(",", join(",", alicloud_vswitch.vswitches.*.id)), count.index%length(split(",", join(",", alicloud_vswitch.vswitches.*.id))))}"
  snat_ip           = "${alicloud_eip.default.ip_address}"
}

resource "alicloud_cs_managed_kubernetes" "k8s" {
  name = "${var.name}"
  availability_zone = "${data.alicloud_zones.main.zones.0.id}"
  new_nat_gateway = false
  vswitch_ids = ["${length(var.vswitch_ids) > 0 ? element(split(",", join(",", var.vswitch_ids)), count.index%length(split(",", join(",", var.vswitch_ids)))) : length(var.vswitch_cidrs) < 1 ? "" : element(split(",", join(",", alicloud_vswitch.vswitches.*.id)), count.index%length(split(",", join(",", alicloud_vswitch.vswitches.*.id))))}"]
  worker_instance_types = ["${data.alicloud_instance_types.default.instance_types.0.id}"]
  worker_numbers = [2]
  key_name = "${alicloud_key_pair.k8s.key_name}"
  pod_cidr = "172.20.0.0/16"
  service_cidr = "172.21.0.0/20"
  install_cloud_monitor = true
  slb_internet_enabled = true
  worker_disk_category  = "cloud_efficiency"
  worker_disk_size  = "50"
}

Debug Output

alicloud_cs_managed_kubernetes.k8s: Creating...
  availability_zone:           "" => "cn-shenzhen-c"
  install_cloud_monitor:       "" => "true"
  key_name:                    "" => "aliyun_k8s_key_pair"
  name:                        "" => "cluster-k8s"
  name_prefix:                 "" => "Terraform-Creation"
  new_nat_gateway:             "" => "false"
  pod_cidr:                    "" => "172.20.0.0/16"
  security_group_id:           "" => "<computed>"
  service_cidr:                "" => "172.21.0.0/20"
  slb_internet_enabled:        "" => "true"
  vpc_id:                      "" => "<computed>"
  vswitch_ids.#:               "" => "1"
  vswitch_ids.0:               "" => "vsw-wz97ptmmopa17pjrr1bi9"
  worker_disk_category:        "" => "cloud_efficiency"
  worker_disk_size:            "" => "50"
  worker_instance_charge_type: "" => "PostPaid"
  worker_instance_types.#:     "" => "1"
  worker_instance_types.0:     "" => "ecs.n1.small"
  worker_nodes.#:              "" => "<computed>"
  worker_numbers.#:            "" => "1"
  worker_numbers.0:            "" => "2"
alicloud_cs_managed_kubernetes.k8s: Still creating... (10s elapsed)
alicloud_cs_managed_kubernetes.k8s: Still creating... (20s elapsed)

Error: Error applying plan:

1 error(s) occurred:

* alicloud_cs_managed_kubernetes.k8s: 1 error(s) occurred:

* alicloud_cs_managed_kubernetes.k8s: Creating ManagedKubernetes Cluster got an error: &common.Error{ErrorResponse:common.ErrorResponse{Response:common.Response{RequestId:"1155A090-BC2E-4F60-9F05-4068F2134866"}, HostId:"", Code:"ErrKubernetesAuditRoleNotAttach", Message:"service role not attach"}, StatusCode:400}

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

Steps to Reproduce

  1. terraform apply
@xh4n3

This comment has been minimized.

Copy link
Contributor

@xh4n3 xh4n3 commented Apr 22, 2019

Hi @paolomainardi ,

There is an authorization you need to do before creating a cluster.

In short, you could try to create a managed cluster via the web console, the server will run an account pre-check for you. It requires you to authorize container service to access some of your products. After accept it, cancel the cluster creation. Then you can create cluster via terraform.

Please see this doc https://www.alibabacloud.com/help/doc-detail/86483.htm

@paolomainardi

This comment has been minimized.

Copy link
Author

@paolomainardi paolomainardi commented Apr 22, 2019

Thanks a lot, this is exactly what I did then, I forgot to update the issue.
Fun fact: With the same user I was able to create k8s clusters just some hours before.

@xh4n3

This comment has been minimized.

Copy link
Contributor

@xh4n3 xh4n3 commented Apr 23, 2019

@paolomainardi That's great!
Please feel free to report any issues on our k8s clusters via our online support system, where you can get faster responses.

@seitosan

This comment has been minimized.

Copy link

@seitosan seitosan commented Apr 24, 2019

@paolomainardi I had the same issue. I opened a ticket on alicloud support.
The solution are to set policy and role to register the k8s roles

@paolomainardi

This comment has been minimized.

Copy link
Author

@paolomainardi paolomainardi commented Apr 29, 2019

It should be handled by terraform ?

@xiaozhu36

This comment has been minimized.

Copy link
Collaborator

@xiaozhu36 xiaozhu36 commented May 6, 2019

HI @paolomainardi I am sorry to tell you that currently this progress only can be done by the Web Console and there is no api to support. The good new is that this step only need do once.

@xiaozhu36

This comment has been minimized.

Copy link
Collaborator

@xiaozhu36 xiaozhu36 commented May 27, 2019

Closed by no reply.

@xiaozhu36 xiaozhu36 closed this May 27, 2019
@tsujp

This comment has been minimized.

Copy link

@tsujp tsujp commented Jul 24, 2019

This issue should be reopened.

The alicloud_cs_managed_kubernetes resource should handle this automatically, having to create resources and instantly delete them to enable automatic resource deployment is arbitrary, undocumented (except for this issue) and – therefore – unfriendly.

@xh4n3

This comment has been minimized.

Copy link
Contributor

@xh4n3 xh4n3 commented Jul 24, 2019

There's a note on the docs, the error currently cannot be resolved on the client side.

NOTE: You need to activate several other products and confirm Authorization Policy used by Container Service before using this resource. Please refer to the Authorization management and Cluster management sections in the Document Center.
https://www.terraform.io/docs/providers/alicloud/r/cs_managed_kubernetes.html

@tsujp

This comment has been minimized.

Copy link

@tsujp tsujp commented Jul 24, 2019

I've seen the note but it's still arbitrary that permissions must be manually granted for container service and not for other resources. Whatever backend API or change of behaviour needed to fix that should be implemented so that the resource can work predictably would be a nice QoL change.

Failing that an authorization resource so we can set the required permissions programatically.

@leonrodenburg

This comment has been minimized.

Copy link
Contributor

@leonrodenburg leonrodenburg commented Sep 13, 2019

I also ran into this issue. After the console created the role for me I took its contents and put it in Terraform. Then I removed the one created by the console, deployed again and the cluster was created successfully.

You can add the following resources to create the role (note this is 0.12, so if you are still on 0.11, you have to change it a little bit):

resource "alicloud_ram_role" "k8s-audit-role" {
  name     = "AliyunCSKubernetesAuditRole"
  document = <<EOF
  {
    "Statement": [
      {
        "Action": "sts:AssumeRole",
        "Effect": "Allow",
        "Principal": {
          "Service": [
            "cs.aliyuncs.com"
          ]
        }
      }
    ],
    "Version": "1"
  }
  EOF
  description = "Allows Kubernetes to access other cloud resources"
  force = true
}

resource "alicloud_ram_role_policy_attachment" "k8s-audit-policy" {
  role_name = alicloud_ram_role.k8s-audit-role.name

  policy_name = "AliyunCSKubernetesAuditRolePolicy"
  policy_type = "System"
}

You will have to take an explicit dependency on the policy attachment in the cluster, so the role will be created before the cluster is deployed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.