Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce number of calls to EC2 API #909

Merged
merged 1 commit into from
Apr 17, 2020
Merged

Conversation

mogren
Copy link
Contributor

@mogren mogren commented Apr 11, 2020

Issue #, if available: #887

Description of changes:

  • Remove Tags from the metadata struct, since tags are only available by calling EC2
  • Remove currentENIs value since it's not used anywhere
  • Made a separate function to get IPv4 addresses for one ENI from EC2
  • Made another function to only get the ENI AttachmentID from EC2
  • Batch the call to check all attached ENIs into one API request
  • Added tests for no_manage
  • Verified that no_manage tags on both restarts and adding new ENIs to a node

Test showing number of DescribeNetworkInterfaces calls running 5000 pods on 100 nodes:
ipamd-ec2-api-calls

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@mogren mogren force-pushed the less-ec2-calls branch 2 times, most recently from 480a226 to b492bba Compare April 11, 2020 01:53
@mogren mogren requested a review from jaypipes April 11, 2020 19:40
pkg/awsutils/awsutils.go Show resolved Hide resolved
pkg/awsutils/awsutils.go Show resolved Hide resolved
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd_test.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd.go Show resolved Hide resolved
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
@mogren mogren force-pushed the less-ec2-calls branch 3 times, most recently from cd55c21 to dde2d35 Compare April 14, 2020 18:21
@mogren mogren force-pushed the less-ec2-calls branch 5 times, most recently from aa34f6f to df7a23a Compare April 14, 2020 22:37
Copy link
Contributor

@jaypipes jaypipes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few relatively minor suggestions, but this is getting close. Nice work on it, @mogren :)

pkg/awsutils/awsutils.go Outdated Show resolved Hide resolved
pkg/awsutils/awsutils.go Outdated Show resolved Hide resolved
pkg/awsutils/awsutils.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd.go Show resolved Hide resolved
Copy link
Contributor

@jaypipes jaypipes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making those changes, @mogren. 👍 from me

@jaypipes jaypipes merged commit bc571be into aws:master Apr 17, 2020
jaypipes added a commit to jaypipes/amazon-vpc-cni-k8s that referenced this pull request Apr 17, 2020
Practice good code safety in the `EC2MetadataCache.getENIAttachmentID()`
method by not assuming that either the `DescribeNetworkInterfacesOutput`
struct's `NetworkInterfaces` field is not empty and that the first
`NetworkInterface` struct that collection has a non-nil `Attachment`
field.

Fixes Issue aws#914 however note that with aws#909, the source code changed
dramatically and this patch will need to be written differently for
v1.5.x branches.
mogren pushed a commit that referenced this pull request Apr 17, 2020
Practice good code safety in the `EC2MetadataCache.getENIAttachmentID()`
method by not assuming that either the `DescribeNetworkInterfacesOutput`
struct's `NetworkInterfaces` field is not empty and that the first
`NetworkInterface` struct that collection has a non-nil `Attachment`
field.

Fixes Issue #914 however note that with #909, the source code changed
dramatically and this patch will need to be written differently for
v1.5.x branches.
mogren pushed a commit to mogren/amazon-vpc-cni-k8s that referenced this pull request Apr 20, 2020
mogren pushed a commit to mogren/amazon-vpc-cni-k8s that referenced this pull request Apr 20, 2020
Practice good code safety in the `EC2MetadataCache.getENIAttachmentID()`
method by not assuming that either the `DescribeNetworkInterfacesOutput`
struct's `NetworkInterfaces` field is not empty and that the first
`NetworkInterface` struct that collection has a non-nil `Attachment`
field.

Fixes Issue aws#914 however note that with aws#909, the source code changed
dramatically and this patch will need to be written differently for
v1.5.x branches.
mogren pushed a commit that referenced this pull request Apr 20, 2020
mogren pushed a commit that referenced this pull request Apr 20, 2020
Practice good code safety in the `EC2MetadataCache.getENIAttachmentID()`
method by not assuming that either the `DescribeNetworkInterfacesOutput`
struct's `NetworkInterfaces` field is not empty and that the first
`NetworkInterface` struct that collection has a non-nil `Attachment`
field.

Fixes Issue #914 however note that with #909, the source code changed
dramatically and this patch will need to be written differently for
v1.5.x branches.
@mogren mogren deleted the less-ec2-calls branch September 4, 2020 05:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants