inventory/aws_ec2: allow multi-entries for one host #1026
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
goneri
changed the title
Merged
Docs Build 📝Thank you for contribution!✨ This PR has been merged and your docs changes will be incorporated when they are next published. |
ansibullbot
added
WIP
alinabuzachis
pushed a commit
to alinabuzachis/amazon.aws
that referenced
this pull request
Sep 16, 2022
…ns#1026) Add AWSRetry.jittered_backoff to rds_instance_info SUMMARY Add AWSRetry.jittered_backoff to the rds_instance_info module. When calling rds_instance_info we have been seeing API rate limit errors from AWS. When calling this module, it usually runs to about 90-150 times in a minute before we get rate limited. Using jittered_backoff should significantly decrease the number of times we see API rate limits here. 02:20:36 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded 02:20:36 fatal: [polaris -> localhost]: FAILED! => {"boto3_version": "1.20.22", "botocore_version": "1.23.22", "changed": false, "error": {"code": "Throttling", "message": "Rate exceeded", "type": "Sender"}, "msg": "Couldn't get instance information: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded", "response_metadata": {"http_headers": {"connection": "close", "content-length": "254", "content-type": "text/xml", "date": "Tue, 15 Mar 2022 09:20:34 GMT", "x-amzn-requestid": "5de8131e-3f59-4b04-af25-5f7083ee09b9"}, "http_status_code": 400, "max_attempts_reached": true, "request_id": "5de8131e-3f59-4b04-af25-5f7083ee09b9", "retry_attempts": 4}} ISSUE TYPE Bugfix Pull Request COMPONENT NAME rds_instance_info ADDITIONAL INFORMATION Decorated rds_instance_info with AWSRetry.jittered_backoff Reviewed-by: Mark Woolley <mw@marknet15.com> Reviewed-by: Mark Chappell <None> This commit was initially merged in https://github.com/ansible-collections/community.aws See: ansible-collections/community.aws@6a2793a
goneri
force-pushed
the
inventory-aws_ec2-allow-multi-entries-for-one-host_16612
branch
from
8a374f2
to
9ffb35a
Compare
1 task
goneri
force-pushed
the
inventory-aws_ec2-allow-multi-entries-for-one-host_16612
branch
2 times, most recently
from
71e22b6
to
299fd6c
Compare
goneri
changed the title
goneri
force-pushed
the
inventory-aws_ec2-allow-multi-entries-for-one-host_16612
branch
from
299fd6c
to
3c26253
Compare
There was a problem hiding this comment.
@goneri Could you also please update the documentation with this option https://github.com/ansible-collections/amazon.aws/blob/main/docs/docsite/rst/aws_ec2_guide.rst ?
goneri
pushed a commit
to alinabuzachis/amazon.aws
that referenced
this pull request
Sep 21, 2022
…ns#1026) Add AWSRetry.jittered_backoff to rds_instance_info SUMMARY Add AWSRetry.jittered_backoff to the rds_instance_info module. When calling rds_instance_info we have been seeing API rate limit errors from AWS. When calling this module, it usually runs to about 90-150 times in a minute before we get rate limited. Using jittered_backoff should significantly decrease the number of times we see API rate limits here. 02:20:36 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded 02:20:36 fatal: [polaris -> localhost]: FAILED! => {"boto3_version": "1.20.22", "botocore_version": "1.23.22", "changed": false, "error": {"code": "Throttling", "message": "Rate exceeded", "type": "Sender"}, "msg": "Couldn't get instance information: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded", "response_metadata": {"http_headers": {"connection": "close", "content-length": "254", "content-type": "text/xml", "date": "Tue, 15 Mar 2022 09:20:34 GMT", "x-amzn-requestid": "5de8131e-3f59-4b04-af25-5f7083ee09b9"}, "http_status_code": 400, "max_attempts_reached": true, "request_id": "5de8131e-3f59-4b04-af25-5f7083ee09b9", "retry_attempts": 4}} ISSUE TYPE Bugfix Pull Request COMPONENT NAME rds_instance_info ADDITIONAL INFORMATION Decorated rds_instance_info with AWSRetry.jittered_backoff Reviewed-by: Mark Woolley <mw@marknet15.com> Reviewed-by: Mark Chappell <None> This commit was initially merged in https://github.com/ansible-collections/community.aws See: ansible-collections/community.aws@6a2793a
|
recheck |
goneri
force-pushed
the
inventory-aws_ec2-allow-multi-entries-for-one-host_16612
branch
from
2537a21
to
510d80f
Compare
Add an option to allow multiple duplicated entry for on single instance. Closes: ansible-collections#950
This can happen is a host is found but doesnt match any of the `hostnames` rules.
goneri
force-pushed
the
inventory-aws_ec2-allow-multi-entries-for-one-host_16612
branch
from
510d80f
to
a200eee
Compare
jillr
reviewed
Sep 27, 2022
| @@ -176,6 +184,9 @@ | |||
| separator: '-' # Hostname will be aws-test_literal | |||
| prefix: 'aws' | |||
|
|
|||
| # Returns all the hostnames for a given instance | |||
| allow_duplicated_hosts: False | |||
There was a problem hiding this comment.
Suggested change
| allow_duplicated_hosts: False | |
| allow_duplicated_hosts: True |
There was a problem hiding this comment.
My understanding was the the new behaviour is a bug and we would like to switch back to what we had in 4.0.0.
There was a problem hiding this comment.
The fix in this PR #862 should do this (that is how it should be, it's not a bug really).
- Fix returned hosts when
hostnames:
- tag:Tag1,Tag2
- Fix returned hosts when
hostnames:
- tag:Tag1
- tag:Tag2
- Fix returned hosts when
hostnames:
- tag:Tag1=Test1,Tag2=Test2
Given and EC2 instance with the following tags
tags:
Tag1: Test1
Tag2: Test2
Instead of only returning
"aws_ec2": {
"hosts": [
"Test1"
]
},
"tag_Name_instance_02": {
"hosts": [
"Test1"
]
},
"tag_Tag1_Test1": {
"hosts": [
"Test1"
]
},
"tag_Tag2_Test2": {
"hosts": [
"Test1"
]
}
It returns now
{
"_meta": {
"hostvars": {
"Test1": {
"ami_launch_index": 0,
"ansible_host": "10.210.0.101",
...},
"Test2": {
"ami_launch_index": 0,
"ansible_host": "10.210.0.101",
...},
},
"all": {
"children": [
"aws_ec2",
"tag_Name_instance_02",
"tag_Tag1_Test1",
"tag_Tag2_Test2",
"ungrouped"
]
},
"aws_ec2": {
"hosts": [
"Test1",
"Test2"
]
},
"tag_Name_instance_02": {
"hosts": [
"Test1",
"Test2"
]
},
"tag_Tag1_Test1": {
"hosts": [
"Test1",
"Test2"
]
},
"tag_Tag2_Test2": {
"hosts": [
"Test1",
"Test2"
]
}
}
The problem is it altered the way the hosts were returned introducing a breaking change for the users unfortunately. So, for some users this behaviour is useful while for some others it is not. What about @tremble's suggestion here #862 (comment) ?
There was a problem hiding this comment.
This PR implements what @markuman describe here #862 (comment)
There was a problem hiding this comment.
I might just be confused. I would expect allow_duplicated_hosts: False to only return unique names. And I would expect what markuman suggested (unique_hostnames: true) to do the same thing.
In this example the comment reads:
# Returns all the hostnames for a given instance
But the example parameter is allow_duplicated_hosts: False - which I think would not return "all" hostnames, because it would only return unique / non-duplicate names.
Is my logic wrong?
This comment was marked as resolved.
This comment was marked as resolved.
goneri
changed the title
goneri
changed the title
alinabuzachis
approved these changes
Sep 28, 2022
markuman
approved these changes
Sep 28, 2022
1 task
1 task
abikouo
pushed a commit
to abikouo/amazon.aws
that referenced
this pull request
Sep 18, 2023
…ns#1026) Add AWSRetry.jittered_backoff to rds_instance_info SUMMARY Add AWSRetry.jittered_backoff to the rds_instance_info module. When calling rds_instance_info we have been seeing API rate limit errors from AWS. When calling this module, it usually runs to about 90-150 times in a minute before we get rate limited. Using jittered_backoff should significantly decrease the number of times we see API rate limits here. 02:20:36 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded 02:20:36 fatal: [polaris -> localhost]: FAILED! => {"boto3_version": "1.20.22", "botocore_version": "1.23.22", "changed": false, "error": {"code": "Throttling", "message": "Rate exceeded", "type": "Sender"}, "msg": "Couldn't get instance information: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded", "response_metadata": {"http_headers": {"connection": "close", "content-length": "254", "content-type": "text/xml", "date": "Tue, 15 Mar 2022 09:20:34 GMT", "x-amzn-requestid": "5de8131e-3f59-4b04-af25-5f7083ee09b9"}, "http_status_code": 400, "max_attempts_reached": true, "request_id": "5de8131e-3f59-4b04-af25-5f7083ee09b9", "retry_attempts": 4}} ISSUE TYPE Bugfix Pull Request COMPONENT NAME rds_instance_info ADDITIONAL INFORMATION Decorated rds_instance_info with AWSRetry.jittered_backoff Reviewed-by: Mark Woolley <mw@marknet15.com> Reviewed-by: Mark Chappell <None>
abikouo
pushed a commit
to abikouo/amazon.aws
that referenced
this pull request
Sep 18, 2023
…ns#1026) Add AWSRetry.jittered_backoff to rds_instance_info SUMMARY Add AWSRetry.jittered_backoff to the rds_instance_info module. When calling rds_instance_info we have been seeing API rate limit errors from AWS. When calling this module, it usually runs to about 90-150 times in a minute before we get rate limited. Using jittered_backoff should significantly decrease the number of times we see API rate limits here. 02:20:36 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded 02:20:36 fatal: [polaris -> localhost]: FAILED! => {"boto3_version": "1.20.22", "botocore_version": "1.23.22", "changed": false, "error": {"code": "Throttling", "message": "Rate exceeded", "type": "Sender"}, "msg": "Couldn't get instance information: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded", "response_metadata": {"http_headers": {"connection": "close", "content-length": "254", "content-type": "text/xml", "date": "Tue, 15 Mar 2022 09:20:34 GMT", "x-amzn-requestid": "5de8131e-3f59-4b04-af25-5f7083ee09b9"}, "http_status_code": 400, "max_attempts_reached": true, "request_id": "5de8131e-3f59-4b04-af25-5f7083ee09b9", "retry_attempts": 4}} ISSUE TYPE Bugfix Pull Request COMPONENT NAME rds_instance_info ADDITIONAL INFORMATION Decorated rds_instance_info with AWSRetry.jittered_backoff Reviewed-by: Mark Woolley <mw@marknet15.com> Reviewed-by: Mark Chappell <None>
abikouo
pushed a commit
to abikouo/amazon.aws
that referenced
this pull request
Oct 24, 2023
…ns#1026) Add AWSRetry.jittered_backoff to rds_instance_info SUMMARY Add AWSRetry.jittered_backoff to the rds_instance_info module. When calling rds_instance_info we have been seeing API rate limit errors from AWS. When calling this module, it usually runs to about 90-150 times in a minute before we get rate limited. Using jittered_backoff should significantly decrease the number of times we see API rate limits here. 02:20:36 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded 02:20:36 fatal: [polaris -> localhost]: FAILED! => {"boto3_version": "1.20.22", "botocore_version": "1.23.22", "changed": false, "error": {"code": "Throttling", "message": "Rate exceeded", "type": "Sender"}, "msg": "Couldn't get instance information: An error occurred (Throttling) when calling the DescribeDBInstances operation (reached max retries: 4): Rate exceeded", "response_metadata": {"http_headers": {"connection": "close", "content-length": "254", "content-type": "text/xml", "date": "Tue, 15 Mar 2022 09:20:34 GMT", "x-amzn-requestid": "5de8131e-3f59-4b04-af25-5f7083ee09b9"}, "http_status_code": 400, "max_attempts_reached": true, "request_id": "5de8131e-3f59-4b04-af25-5f7083ee09b9", "retry_attempts": 4}} ISSUE TYPE Bugfix Pull Request COMPONENT NAME rds_instance_info ADDITIONAL INFORMATION Decorated rds_instance_info with AWSRetry.jittered_backoff Reviewed-by: Mark Woolley <mw@marknet15.com> Reviewed-by: Mark Chappell <None>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Depends-On: #1066
Add an option to allow multiple duplicated entries for one single instance.
Closes: #950