Optimize allocation of ENI

[SaranBalaji90]

Ipamd allocates ENI based on IP Pool size. But it doesn't consider instance based limits specified in https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html. Therefore even when the available IP Pool size is reaching its maximum value on a specific instance class, Ipamd tries to allocate ENI continuously. This loop runs every 5 seconds once and invokes 3 EC2 APIs (Create, Attach and Delete)

2018-02-27T13:58:33Z [DEBUG] IP pool stats: total=15, used=10, c.currentMaxAddrsPerENI =6, c.maxAddrsPerENI = 6
2018-02-27T13:58:33Z [INFO] Created a new eni: eni-a7b80f83
2018-02-27T13:58:33Z [DEBUG] Trying to tag newly created eni: keys=k8s-eni-key, value=i-074705deafbe739c0
2018-02-27T13:58:34Z [DEBUG] Tag the newly created eni with arn: arn:aws:ec2:us-west-2:908176817140:network-interface/eni-a7b80f83
2018-02-27T13:58:34Z [DEBUG] Discovered device number is used: 1
2018-02-27T13:58:34Z [DEBUG] Discovered device number is used: 0
2018-02-27T13:58:34Z [DEBUG] Discovered device number is used: 2
2018-02-27T13:58:34Z [DEBUG] Found a free device number: 3
2018-02-27T13:58:34Z [INFO] Exceeded instance eni attachment limit: 3
2018-02-27T13:58:34Z [ERROR] Failed to attach eni eni-a7b80f83: AttachmentLimitExceeded: Interface count 4 exceeds the limit for t2.medium
status code: 400, request id: 42da274d-53a5-43bb-90a4-eaa4cc5a94a2
2018-02-27T13:58:34Z [DEBUG] Trying to delete eni: eni-a7b80f83
2018-02-27T13:58:34Z [INFO] Successfully deleted eni: eni-a7b80f83
2018-02-27T13:58:34Z [ERROR] Failed to increase pool size due to not able to allocate ENI allocate eni: error attaching eni: failed to attach eni: AttachmentLimitExceeded: Interface count 4 exceeds the limit for t2.medium
status code: 400, request id: 42da274d-53a5-43bb-90a4-eaa4cc5a94a2
2018-02-27T13:58:39Z [DEBUG] IP pool stats: total=15, used=10, c.currentMaxAddrsPerENI =6, c.maxAddrsPerENI = 6
2018-02-27T13:58:40Z [INFO] Created a new eni: eni-c6b90ee2
2018-02-27T13:58:40Z [DEBUG] Trying to tag newly created eni: keys=k8s-eni-key, value=i-074705deafbe739c0
2018-02-27T13:58:40Z [DEBUG] Tag the newly created eni with arn: arn:aws:ec2:us-west-2:908176817140:network-interface/eni-c6b90ee2
2018-02-27T13:58:40Z [DEBUG] Discovered device number is used: 1
2018-02-27T13:58:40Z [DEBUG] Discovered device number is used: 0
2018-02-27T13:58:40Z [DEBUG] Discovered device number is used: 2
2018-02-27T13:58:40Z [DEBUG] Found a free device number: 3
2018-02-27T13:58:40Z [INFO] Exceeded instance eni attachment limit: 3
2018-02-27T13:58:40Z [ERROR] Failed to attach eni eni-c6b90ee2: AttachmentLimitExceeded: Interface count 4 exceeds the limit for t2.medium
status code: 400, request id: 7978c341-ba82-426a-9eb8-d616b558891f
2018-02-27T13:58:40Z [DEBUG] Trying to delete eni: eni-c6b90ee2
2018-02-27T13:58:40Z [INFO] Successfully deleted eni: eni-c6b90ee2
2018-02-27T13:58:40Z [ERROR] Failed to increase pool size due to not able to allocate ENI allocate eni: error attaching eni: failed to attach eni: AttachmentLimitExceeded: Interface count 4 exceeds the limit for t2.medium
status code: 400, request id: 7978c341-ba82-426a-9eb8-d616b558891f
2018-02-27T13:58:45Z [DEBUG] IP pool stats: total=15, used=10, c.currentMaxAddrsPerENI =6, c.maxAddrsPerENI = 6
2018-02-27T13:58:46Z [INFO] Created a new eni: eni-fbbb0cdf
2018-02-27T13:58:46Z [DEBUG] Trying to tag newly created eni: keys=k8s-eni-key, value=i-074705deafbe739c0
2018-02-27T13:58:46Z [DEBUG] Tag the newly created eni with arn: arn:aws:ec2:us-west-2:908176817140:network-interface/eni-fbbb0cdf
2018-02-27T13:58:46Z [DEBUG] Discovered device number is used: 1
2018-02-27T13:58:46Z [DEBUG] Discovered device number is used: 0
2018-02-27T13:58:46Z [DEBUG] Discovered device number is used: 2
2018-02-27T13:58:46Z [DEBUG] Found a free device number: 3
2018-02-27T13:58:46Z [INFO] Exceeded instance eni attachment limit: 3

Number of ENIs allocated today:

amazon-vpc-cni-k8s/pkg/awsutils/awsutils.go

Line 53 in eb8fc86

maxENIs = 128

[liwenwu-amazon]

Along with #65, the code has added a table which contains the ENIs limit for each known instance type. And the code also use this table to avoid allocating extra ENIs.