Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ECS DescribeServices max service limit #828

Merged
merged 2 commits into from Apr 26, 2022

Conversation

Rickerd0613
Copy link
Contributor

ECS has a max limit of 10 services per call, not 100. When more than 10 services is in a cluster, the existing method produces a stack trace. https://docs.aws.amazon.com/cli/latest/reference/ecs/describe-services.html

Fixes:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/cartography/sync.py", line 74, in run
    stage_func(neo4j_session, config)
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 117, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/__init__.py", line 225, in start_aws_ingestion
    requested_syncs,
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/__init__.py", line 163, in _sync_multiple_accounts
    aws_requested_syncs=aws_requested_syncs,  # Could be replaced later with per-account requested syncs
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/__init__.py", line 60, in _sync_one_account
    RESOURCE_FUNCTIONS[func_name](**sync_args)
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 117, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/ecs.py", line 571, in sync
    region,
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 117, in timed
    return method(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/util.py", line 146, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/cartography/intel/aws/ecs.py", line 76, in get_ecs_services
    services=service_arn_chunk,
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 415, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.7/site-packages/botocore/client.py", line 745, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the DescribeServices operation: service names can have at most 10 items.

ECS has a max limit of 10 services per call, not 100. When more than 10 services is in a cluster, the existing method stacktraces. https://docs.aws.amazon.com/cli/latest/reference/ecs/describe-services.html
@achantavy
Copy link
Contributor

Looks good to me, thanks for submitting! Can you please sign the CLA?

@Rickerd0613
Copy link
Contributor Author

Huh. Maybe the CLA check needs to re-run? I signed the CLA about 30 minutes after submitting the PR.

@achantavy
Copy link
Contributor

Sometimes the CLA check is flaky - can you close and reopen this PR to restart it?

@Rickerd0613
Copy link
Contributor Author

No problem! Closing

@Rickerd0613 Rickerd0613 reopened this Apr 26, 2022
@Rickerd0613
Copy link
Contributor Author

Success!

Copy link
Contributor

@ramonpetgrave64 ramonpetgrave64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@ramonpetgrave64 ramonpetgrave64 merged commit f16d8db into lyft:master Apr 26, 2022
@Rickerd0613 Rickerd0613 deleted the patch-1 branch April 26, 2022 16:20
shinya-murakami-paypay pushed a commit to paypay/cartography that referenced this pull request May 23, 2022
ECS has a max limit of 10 services per call, not 100. When more than 10 services is in a cluster, the existing method stacktraces. https://docs.aws.amazon.com/cli/latest/reference/ecs/describe-services.html

Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants