Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Excessive calls to AWS get_role_tags; region awareness #1094

Open
achantavy opened this issue Jan 21, 2023 · 0 comments
Open

Excessive calls to AWS get_role_tags; region awareness #1094

achantavy opened this issue Jan 21, 2023 · 0 comments
Labels
AWS Related to cartography's AWS module bug Something isn't working

Comments

@achantavy
Copy link
Contributor

achantavy commented Jan 21, 2023

Description:

What issue is being seen? Describe what should be happening instead of the bug, for example: Cartography should not crash, the expected value isn't returned, the data schema is wrong, etc.

The AWS resourcegroupstaggingapi sync right now runs once per region, which makes sense for resource discovery since most of these resources are region-specific.
However iam roles are global (since the arn of a role does not have a region in it), so our workaround to call get_role_tags will actually be called once per region, which makes things quite slow (a full run of us-east-1 actually timed out on my laptop's boto3 session).

What we should do

  1. We should find a way to avoid wasting API calls to get_role_tags. Maybe this can involve storing or calculating information about which resources are global versus regional.
  2. It does not make sense for AWSTag nodes to have a region as one of their properties so we should remove region from the node.

Please complete the following information::

  • Cartography release version or commit hash [e.g. 0.12.0 or 95e8e11]

0.72.1

@achantavy achantavy added the bug Something isn't working label Jan 21, 2023
@achantavy achantavy added the AWS Related to cartography's AWS module label Jul 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AWS Related to cartography's AWS module bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant