autoscaling_group_info - Fix ValidationError when describing ASGs that have more than 20 Load Balancer Target Groups attached

[bryanhong]

SUMMARY

When retrieving autoscaling group info from an ASG that has more than 20 load balancer target groups attached to it the AWS API returns a ValidationError via Ansible:

    "error": {
        "code": "ValidationError",
        "message": "You cannot describe more than '20' target groups at a time",
        "type": "Sender"
    },

This commit (if length of target_group_arns is greater than 20) breaks the target_group_arns list into chunks and builds the target_group_names list incrementally to avoid exceeding the AWS limit of 20.

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

autoscaling_group_info

ADDITIONAL INFORMATION

Before

An exception occurred during task execution. To see the full traceback, use -vvv. The error was: botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the DescribeTargetGroups operation: You cannot describe more than '20' target groups at a time
failed: [us-west-1] (item=['redacted', 'redacted']) => {
    "ansible_loop_var": "item",
    "boto3_version": "1.26.136",
    "botocore_version": "1.29.146",
    "changed": false,
    "error": {
        "code": "ValidationError",
        "message": "You cannot describe more than '20' target groups at a time",
        "type": "Sender"
    },
    "item": [
        "01",
        "proxy"
    ],
    "response_metadata": {
        "http_headers": {
            "connection": "close",
            "content-length": "321",
            "content-type": "text/xml",
            "date": "Fri, 02 Jun 2023 21:45:34 GMT",
            "x-amzn-requestid": "32205cd9-0c1c-4c8b-bf6f-0536d1251f72"
        },
        "http_status_code": 400,
        "request_id": "32205cd9-0c1c-4c8b-bf6f-0536d1251f72",
        "retry_attempts": 0
    }
}

MSG:

Failed to describe Target Groups: An error occurred (ValidationError) when calling the DescribeTargetGroups operation: You cannot describe more than '20' target groups at a time

These are the sections of output below
https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_info_module.html#return-target_group_arns
https://docs.ansible.com/ansible/latest/collections/amazon/aws/autoscaling_group_info_module.html#return-target_group_names

After this commit, the module is able to retrieve and return target group names vs ValidationError above.

...
            "target_group_arns": [
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs0-001-pd01/54a21ec73d9567de",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs1-001-pd01/9c3c288759270055",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs10-001-pd01/1a8f765efa2b21f3",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs11-001-pd01/be79ff24e7c9dfc8",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs12-001-pd01/b0efa4c98c3707ea",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs13-001-pd01/5ba3bd3e820189e5",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs14-001-pd01/9eb427e64e64cb34",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs15-001-pd01/cd57f2fc112ef1be",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs16-001-pd01/7c96ab6cf57065d9",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs17-001-pd01/b2f2cb1f8c56fccd",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs18-001-pd01/48d5b7e958256383",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs19-001-pd01/c31c2835b325053e",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs2-001-pd01/088e22de058a5682",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs20-001-pd01/06bc6d5594c400d8",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs21-001-pd01/d2e19039fb58901a",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs22-001-pd01/3e542166d868cd51",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs23-001-pd01/915a38152472f256",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs24-001-pd01/0d490d00340a3b0c",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs25-001-pd01/4da43adf85c2c74f",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs26-001-pd01/dd6eed2cf2b3b40a",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs27-001-pd01/b6a70661615b58e5",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs28-001-pd01/9f90bb0063a399d3",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs29-001-pd01/a5de186522047a38",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs3-001-pd01/1abe1864526da4f4",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs30-001-pd01/acdd881d550190b0",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs31-001-pd01/e2b2d9f7496a1e42",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs32-001-pd01/c67d2beab9937b36",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs33-001-pd01/1a6e069d76d35129",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs34-001-pd01/7e7a041ba28d9ba1",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs35-001-pd01/7e9c6316c8bfbdcc",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs36-001-pd01/a63ef453fb2a9ffc",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs37-001-pd01/6d2b70df56a6e6b8",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs38-001-pd01/c29789ed0c5f185a",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs39-001-pd01/b2ee2dd578012057",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs4-001-pd01/6bac5695356d7a31",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs40-001-pd01/7b7a1b4110b676a0",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs41-001-pd01/69f80d314ea3998c",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs42-001-pd01/f3b542db69b6364e",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs43-001-pd01/e2af6db68a4f90c1",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs44-001-pd01/d8e30df7de550115",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs45-001-pd01/28030e72b5771b95",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs46-001-pd01/4c6558b79bea34e9",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs47-001-pd01/8de7ffe13bb7fdd8",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs48-001-pd01/02d57221242667e8",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs49-001-pd01/5847cbf10c46b9aa",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs5-001-pd01/82828f534fc4f0ba",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs6-001-pd01/c6258cfe31de8ff8",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs7-001-pd01/e665c3b1fceb2717",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs8-001-pd01/5309914c19f919cc",
                "arn:aws:elasticloadbalancing:us-west-1:redacted:targetgroup/tgcs9-001-pd01/bf33d7ca75fe4c37"
            ],
            "target_group_names": [
                "tgcs0-001-pd01",
                "tgcs1-001-pd01",
                "tgcs10-001-pd01",
                "tgcs11-001-pd01",
                "tgcs12-001-pd01",
                "tgcs13-001-pd01",
                "tgcs14-001-pd01",
                "tgcs15-001-pd01",
                "tgcs16-001-pd01",
                "tgcs17-001-pd01",
                "tgcs18-001-pd01",
                "tgcs19-001-pd01",
                "tgcs2-001-pd01",
                "tgcs20-001-pd01",
                "tgcs21-001-pd01",
                "tgcs22-001-pd01",
                "tgcs23-001-pd01",
                "tgcs24-001-pd01",
                "tgcs25-001-pd01",
                "tgcs26-001-pd01",
                "tgcs27-001-pd01",
                "tgcs28-001-pd01",
                "tgcs29-001-pd01",
                "tgcs3-001-pd01",
                "tgcs30-001-pd01",
                "tgcs31-001-pd01",
                "tgcs32-001-pd01",
                "tgcs33-001-pd01",
                "tgcs34-001-pd01",
                "tgcs35-001-pd01",
                "tgcs36-001-pd01",
                "tgcs37-001-pd01",
                "tgcs38-001-pd01",
                "tgcs39-001-pd01",
                "tgcs4-001-pd01",
                "tgcs40-001-pd01",
                "tgcs41-001-pd01",
                "tgcs42-001-pd01",
                "tgcs43-001-pd01",
                "tgcs44-001-pd01",
                "tgcs45-001-pd01",
                "tgcs46-001-pd01",
                "tgcs47-001-pd01",
                "tgcs48-001-pd01",
                "tgcs49-001-pd01",
                "tgcs5-001-pd01",
                "tgcs6-001-pd01",
                "tgcs7-001-pd01",
                "tgcs8-001-pd01",
                "tgcs9-001-pd01"
            ],
...

[softwarefactory-project-zuul]

Build failed.
https://ansible.softwarefactory-project.io/zuul/buildset/fd771ff0913f40e38a56ee77cd5f601b

✔️ ansible-galaxy-importer SUCCESS in 4m 08s
✔️ build-ansible-collection SUCCESS in 19m 53s
✔️ ansible-test-splitter SUCCESS in 5m 11s
❌ integration-amazon.aws-1 TIMED_OUT in 1h 00m 39s
Skipped 43 jobs

[softwarefactory-project-zuul]

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/08ba0e6f82ba48b4bd57f7dc6a1242a7

✔️ ansible-galaxy-importer SUCCESS in 3m 59s
✔️ build-ansible-collection SUCCESS in 12m 52s
✔️ ansible-test-splitter SUCCESS in 4m 48s
✔️ integration-amazon.aws-1 SUCCESS in 36m 49s
Skipped 43 jobs

[tremble]

Thanks for taking the time to submit this PR. I'm happy that this code would work, but have some suggestions if you're open to them.

Have you had a look to see if the autoscaling_group module's subject to the same issue?

[bryanhong]

Thanks for taking the time to submit this PR. I'm happy that this code would work, but have some suggestions if you're open to them.

Have you had a look to see if the autoscaling_group module's subject to the same issue?

I hadn't, but it does appear to have a similar section that would hit the problem, proposed change in 5fa66e3

[softwarefactory-project-zuul]

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/b5faef29ab874263a8c34ada890e273b

✔️ ansible-galaxy-importer SUCCESS in 3m 29s
✔️ build-ansible-collection SUCCESS in 12m 36s
✔️ ansible-test-splitter SUCCESS in 5m 06s
✔️ integration-amazon.aws-1 SUCCESS in 26m 56s
Skipped 43 jobs

[tremble]

LGTM, I've copied your changelog entry from autoscaling_group_info for autoscaling_group (we should have an entry for both).

[softwarefactory-project-zuul]

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/d5f0b9a9d2454966b9e6916aac636eee

✔️ ansible-galaxy-importer SUCCESS in 3m 45s
✔️ build-ansible-collection SUCCESS in 14m 36s
✔️ ansible-test-splitter SUCCESS in 4m 39s
✔️ integration-amazon.aws-1 SUCCESS in 32m 14s
Skipped 43 jobs

[softwarefactory-project-zuul]

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/5d0cbf8db40d404383a55fe54fafc0e3

✔️ ansible-galaxy-importer SUCCESS in 4m 14s
✔️ build-ansible-collection SUCCESS in 14m 16s
✔️ ansible-test-splitter SUCCESS in 5m 02s
✔️ integration-amazon.aws-1 SUCCESS in 38m 28s
Skipped 43 jobs

[patchback]

Backport to stable-5: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 3d045fd on top of patchback/backports/stable-5/3d045fda5df9dd6d867c05c29982a6186d616b5a/pr-1593

Backporting merged PR #1593 into main

1. Ensure you have a local repo clone of your fork. Unless you cloned it
   from the upstream, this would be your origin remote.
2. Make sure you have an upstream repo added as a remote too. In these
   instructions you'll refer to it by the name upstream. If you don't
   have it, here's how you can add it:

   $ git remote add upstream https://github.com/ansible-collections/amazon.aws.git

3. Ensure you have the latest copy of upstream and prepare a branch
   that will hold the backported code:

   $ git fetch upstream
   $ git checkout -b patchback/backports/stable-5/3d045fda5df9dd6d867c05c29982a6186d616b5a/pr-1593 upstream/stable-5

4. Now, cherry-pick PR autoscaling_group_info - Fix ValidationError when describing ASGs that have more than 20 Load Balancer Target Groups attached #1593 contents into that branch:

   $ git cherry-pick -x 3d045fda5df9dd6d867c05c29982a6186d616b5a

   If it'll yell at you with something like fatal: Commit 3d045fda5df9dd6d867c05c29982a6186d616b5a is a merge but no -m option was given., add -m 1 as follows instead:

   $ git cherry-pick -m1 -x 3d045fda5df9dd6d867c05c29982a6186d616b5a

5. At this point, you'll probably encounter some merge conflicts. You must
   resolve them in to preserve the patch from PR autoscaling_group_info - Fix ValidationError when describing ASGs that have more than 20 Load Balancer Target Groups attached #1593 as close to the
   original as possible.
6. Push this branch to your fork on GitHub:

   $ git push origin patchback/backports/stable-5/3d045fda5df9dd6d867c05c29982a6186d616b5a/pr-1593

7. Create a PR, ensure that the CI is green. If it's not — update it so that
   the tests and any other checks pass. This is it!
   Now relax and wait for the maintainers to process your pull request
   when they have some cycles to do reviews. Don't worry — they'll tell you if
   any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

[patchback]

Backport to stable-6: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-6/3d045fda5df9dd6d867c05c29982a6186d616b5a/pr-1593

Backported as #1596

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

[bryanhong]

Thanks @tremble and @alinabuzachis for the guidance and help getting this in, much appreciated.