AWS CCM not working - missing ProviderID

[tomerleib]

RKE version:
1.2.3
Docker version: (docker version,docker info preferred)

Client: Docker Engine - Community
 Version:           19.03.14
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        5eb3275d40
 Built:             Tue Dec  1 19:20:17 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.14
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       5eb3275d40
  Built:            Tue Dec  1 19:18:45 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Operating system and kernel: (cat /etc/os-release, uname -r preferred)

Ubuntu 18.04
5.4.0-1029-aws

Type/provider of hosts: (VirtualBox/Bare-metal/AWS/GCE/DO)
AWS
cluster.yml file:

nodes:
    - address: xxx.xxx.xxx.xxx
      hostname_override: ec2-xxx-xxx-xxx-xxx.us-east-2.compute.amazonaws.com
      user: ubuntu
      role:
        - controlplane
        - etcd
        - worker
    - address: xxx.xxx.xxx.xxx
      hostname_override: ec2-xxx-xxx-xxx-xxx.us-east-2.compute.amazonaws.com
      user: ubuntu
      role:
        - controlplane
        - etcd
        - worker
    - address: xxx.xxx.xxx.xxx
      hostname_override: ec2-xxx-xxx-xxx-xxx.us-east-2.compute.amazonaws.com
      user: ubuntu
      role:
        - controlplane
        - etcd
        - worker 
cluster_name: rke
kubernetes_version: v1.19.4-rancher1-1
cloud_provider:
    name: external

Steps to Reproduce:

• Install cloud-provider-aws out-of-tree from https://kubernetes.github.io/cloud-provider-aws/getting_started.html
• Validate the pods are running
• Create an ELB

kind: Service
apiVersion: v1
metadata:
  name: hello
spec:
  type: LoadBalancer
  selector:
    app: hello
  ports:
    - name: http
      protocol: TCP
      # ELB's port
      port: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:
        - name: hello
          image: nginx

Results:
ELB is successfully created, but the nodes aren't registered.
In the logs I've found the following errors:

E0106 10:14:13.614080       1 node_controller.go:370] failed to initialize node ec2-xxx-xxx-xxx-xxx.us-east-2.compute.amazonaws.com at cloudprovider: failed to get instance ID from cloud provider: instance not found
...
W0106 10:19:23.217599       1 instances.go:115] node "ec2-xxx-xxx-xxx-xxx.us-east-2.compute.amazonaws.com" did not have ProviderID set

For a single node, I can add providerid as:

services:
    kube-controller:
      extra_args: provider-id=aws:///xxx/xxxxx

However, for a multinode, it won't work since each node have it's own ID.
Any idea how it can work out or stick with in-tree AWS?

[stale]

This issue/PR has been automatically marked as stale because it has not had activity (commit/comment/label) for 60 days. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[atsai1220]

Running into this issue too.

[beergeek]

Was there any way to solve this for a multi-node cluster? I am having the exact issue.

[beergeek]

I got this working by creating spec.providerID: aws:///<REGION>/<INSTANCEID> on each worker node