Skip to content

Conversation

@nckturner
Copy link
Contributor

@nckturner nckturner commented Sep 17, 2021

  • EC2 pagination doesn't work if MaxResults is not set
  • Only set if InstanceIds isn't set.

What type of PR is this?

Uncomment only one, leave it on its own line:

/kind bug

What this PR does / why we need it:
The describeInstances API doesn't paginate unless maxResults is set.
Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:
/hold

Does this PR introduce a user-facing change?:

Ensure pagination of EC2 describeInstances calls

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. release-note Denotes a PR that will be considered when it comes time to generate release notes. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Sep 17, 2021
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 17, 2021
@wongma7
Copy link

wongma7 commented Sep 17, 2021

kubernetes/kubernetes#102927 this updated the cache to have a ttl of just 10 minutes. Seems like a very bad idea if we are trying to optimize our calls.

* EC2 pagination doesn't work if MaxResults is not set.
* Only set if InstanceIds isn't set.
@nckturner nckturner force-pushed the paginate-describe-instances branch from 9c63fa9 to 431d0fb Compare September 17, 2021 23:20
@nckturner
Copy link
Contributor Author

/unhold

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 17, 2021
@nckturner nckturner changed the title Check if MaxResults is not set Set MaxResults if it is not set Sep 17, 2021
@wongma7
Copy link

wongma7 commented Sep 17, 2021

do you mind double checking that describeAllInstancesUncached

func (c *instanceCache) describeAllInstancesUncached() (*allInstancesSnapshot, error) {
ultimately calls this function? I am confident in my ctrl+clicking skills but just want to make 100% sure that we're covering all calls given that there are quite a few layers (describeAllInstancesUncached, describeInstances lowercase, DescribeInstances uppercase, sdk go's DescribeInstances uppercase...

@wongma7
Copy link

wongma7 commented Sep 18, 2021

describeAllInstancesUncached calls c.cloud.describeInstances (c is for cache)

instances, err := c.cloud.describeInstances(filters)

func (c *Cloud) describeInstances(filters []*ec2.Filter) ([]*ec2.Instance, error) {

c.cloud.describeInstances calls c.ec2.DescribeInstances (now c is for cloud)
response, err := c.ec2.DescribeInstances(request)

cloud.ec2 is a v1.EC2 interface
type EC2 interface {

and the "real"/non-mock implementation of it by awsSdkEC2 is
func (s *awsSdkEC2) DescribeInstances(request *ec2.DescribeInstancesInput) ([]*ec2.Instance, error) {

which calls s.ec2.DescribeInstances (now ec2 is an aws-sdk-go ec2.EC2 not a v1.EC2/awsSdkEC2)

@wongma7
Copy link

wongma7 commented Sep 18, 2021

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 18, 2021
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nckturner, wongma7

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants