Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws.elasticsearch Error Scanning More Than 5 domains #5793

Closed
jtroberts83 opened this issue May 22, 2020 · 3 comments · Fixed by #5796
Closed

aws.elasticsearch Error Scanning More Than 5 domains #5793

jtroberts83 opened this issue May 22, 2020 · 3 comments · Fixed by #5796
Assignees
Labels

Comments

@jtroberts83
Copy link
Contributor

Describe the bug
When running any elasticsearch policy on an account region with more than 5 elasticsearch domains the policy now bombs out with the error - error:An error occurred (ValidationException) when calling the DescribeElasticsearchDomains operation: Please provide a maximum of 5 Elasticsearch domain names to describe.

To Reproduce
Create 6 es domains and run an elasticsearch c7n policy, error will occur

Expected behavior
It should chunk the calls into domains of 5 or less

Background (please complete the following information):

  • OS: Ubuntu v20
  • Python Version: 3.8
  • Custodian Version: 0.9.2.0
  • Tool Version: [if applicable]
  • Cloud Provider: aws
  • Policy: any policy which queries ES
  • Traceback:
[ERROR]	2020-05-22T14:51:25.978Z	9ef7929b-b494-434e-9f9f-dfdfdfdfdfdfd Error while executing policy
Traceback (most recent call last):
  File "/var/task/c7n/policy.py", line 291, in run
    resources = self.policy.resource_manager.resources()
  File "/var/task/c7n/query.py", line 466, in resources
    resources = self.augment(resources)
  File "/var/task/c7n/query.py", line 521, in augment
    return self.source.augment(resources)
  File "/var/task/c7n/resources/elasticsearch.py", line 48, in augment
    return _augment(domains)
  File "/var/task/c7n/resources/elasticsearch.py", line 39, in _augment
    resources = self.manager.retry(
  File "/var/task/c7n/utils.py", line 373, in _retry
    return func(*args, **kw)
  File "/var/task/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/var/task/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the DescribeElasticsearchDomains operation: Please provide a maximum of 5 Elasticsearch domain names to describe.
  • custodian version --debug output

Additional context
Seems to be introduced with 0.9.2.0

@PratMis
Copy link
Collaborator

PratMis commented May 26, 2020

We're also running into this issue. thanks for reporting @jtroberts83

@kapilt
Copy link
Collaborator

kapilt commented May 26, 2020

fix merged, I'll queue up a micro release

@PratMis
Copy link
Collaborator

PratMis commented May 26, 2020

Thanks @kapilt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants