Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EC2 autoscaler: avoid hitting aws filter limits #1954

Merged
merged 1 commit into from
Nov 13, 2015

Conversation

xvrl
Copy link
Member

@xvrl xvrl commented Nov 11, 2015

Prevents the following error below from happening.
As a side benefit this also materialized the instanceId list, so it should reduce garbage creation.

com.amazonaws.AmazonServiceException: The maximum number of filter values specified on a single call is 200 (Service: AmazonEC2; Status Code: 400; Error Code: FilterLimitExceeded; Request ID: XXXXX)

@@ -45,6 +46,7 @@
public class EC2AutoScaler implements AutoScaler<EC2EnvironmentConfig>
{
private static final EmittingLogger log = new EmittingLogger(EC2AutoScaler.class);
public static final int MAX_AWS_FILTER_VALUES = 100;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this documented by AWS somewhere? if so can we put a comment where this came from?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't searched the docs, but the limit appears to be 200 by default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also add a comment ?

@drcrallen
Copy link
Contributor

👍

1 similar comment
@nishantmonu51
Copy link
Member

👍

@xvrl
Copy link
Member Author

xvrl commented Nov 11, 2015

hold on before merging, there's another place where we might hit this limit

@drcrallen
Copy link
Contributor

Still 👍

@xvrl
Copy link
Member Author

xvrl commented Nov 11, 2015

ok I added comment, and fix the other filter query as well, please have a look

@drcrallen
Copy link
Contributor

I'm guessing there's no way to check this in a unit test is there?

@xvrl
Copy link
Member Author

xvrl commented Nov 12, 2015

@drcrallen I can add a unit test, sure,

@xvrl
Copy link
Member Author

xvrl commented Nov 12, 2015

added tests

@nishantmonu51
Copy link
Member

👍

@xvrl xvrl added this to the 0.8.3 milestone Nov 12, 2015
drcrallen added a commit that referenced this pull request Nov 13, 2015
EC2 autoscaler: avoid hitting aws filter limits
@drcrallen drcrallen merged commit 46527a9 into apache:master Nov 13, 2015
@drcrallen drcrallen deleted the fix-stupid-aws-limit branch November 13, 2015 18:52
@xvrl xvrl mentioned this pull request Dec 1, 2015
@gianm gianm mentioned this pull request Dec 4, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants