ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1575

alinabuzachis · 2023-05-26T11:24:45Z

SUMMARY

Handle decompression when user-data is compressed. The fetch_url method from ansible.module_utils.urls does not decompress the user-data because the header is missing (The API does not set 'Content-Encoding' = 'gzip').

ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME

ec2_metadata_facts

ADDITIONAL INFORMATION

…od from ansible.module_utils.urls does not decompress the user-data unless the header explicitly contains 'Content-Encoding' = 'gzip'. Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

plugins/modules/ec2_metadata_facts.py

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

gravesm · 2023-05-26T12:26:15Z

plugins/modules/ec2_metadata_facts.py

+        try:
+            if is_compressed:
+                # Data is compressed, attempt decompression and decode using UTF-8
+                decoded_data = zlib.decompress(data, zlib.MAX_WBITS | 32).decode("utf-8")


If the decompression succeeds, but the decoding fails with a UnicodeDecodeError, then the except UnicodeDecodeError block is going to fail because it will try to decode compressed data. The easiest way to handle this is probably with two try/except blocks, one to decompress the data, and the other to decode it.

softwarefactory-project-zuul · 2023-05-26T12:42:39Z

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/27aa20e4517144f189a209475d6d70c2

✔️ ansible-galaxy-importer SUCCESS in 4m 10s
✔️ build-ansible-collection SUCCESS in 13m 14s
✔️ ansible-test-splitter SUCCESS in 5m 12s
✔️ integration-amazon.aws-1 SUCCESS in 9m 29s
Skipped 43 jobs

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

softwarefactory-project-zuul · 2023-05-26T13:16:49Z

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/03133cca2f13459eaf67377285731bf1

✔️ ansible-galaxy-importer SUCCESS in 3m 38s
✔️ build-ansible-collection SUCCESS in 13m 30s
✔️ ansible-test-splitter SUCCESS in 4m 49s
✔️ integration-amazon.aws-1 SUCCESS in 10m 29s
Skipped 43 jobs

plugins/modules/ec2_metadata_facts.py

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

changelogs/fragments/20230526-ec2_mertadata_facts-handle_compressed_user_data.yml

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

changelogs/fragments/20230526-ec2_mertadata_facts-handle_compressed_user_data.yml

softwarefactory-project-zuul · 2023-05-30T11:18:07Z

Build succeeded.
https://ansible.softwarefactory-project.io/zuul/buildset/0eb4f03a47ef421fa8f8272d26c115c8

✔️ ansible-galaxy-importer SUCCESS in 4m 04s
✔️ build-ansible-collection SUCCESS in 13m 01s
✔️ ansible-test-splitter SUCCESS in 4m 40s
✔️ integration-amazon.aws-1 SUCCESS in 8m 46s
Skipped 43 jobs

softwarefactory-project-zuul · 2023-05-30T13:23:30Z

Build succeeded (gate pipeline).
https://ansible.softwarefactory-project.io/zuul/buildset/cd0660b599bb436e90cefbd9103c37d1

✔️ ansible-galaxy-importer SUCCESS in 5m 12s
✔️ build-ansible-collection SUCCESS in 12m 36s
✔️ ansible-test-splitter SUCCESS in 4m 54s
✔️ integration-amazon.aws-1 SUCCESS in 13m 16s
Skipped 43 jobs

patchback · 2023-05-30T13:23:34Z

Backport to stable-5: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply f1fe603 on top of patchback/backports/stable-5/f1fe603084fdf05b70a856ea137a608eb663ccb9/pr-1575

Backporting merged PR #1575 into main

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/ansible-collections/amazon.aws.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/stable-5/f1fe603084fdf05b70a856ea137a608eb663ccb9/pr-1575 upstream/stable-5

Now, cherry-pick PR ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1575 contents into that branch:
```
$ git cherry-pick -x f1fe603084fdf05b70a856ea137a608eb663ccb9
```
If it'll yell at you with something like fatal: Commit f1fe603084fdf05b70a856ea137a608eb663ccb9 is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x f1fe603084fdf05b70a856ea137a608eb663ccb9
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1575 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/stable-5/f1fe603084fdf05b70a856ea137a608eb663ccb9/pr-1575

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2023-05-30T13:23:41Z

Backport to stable-6: 💚 backport PR created

✅ Backport PR branch: patchback/backports/stable-6/f1fe603084fdf05b70a856ea137a608eb663ccb9/pr-1575

Backported as #1580

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

… is compressed (#1575) ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed SUMMARY Handle decompression when user-data is compressed. The fetch_url method from ansible.module_utils.urls does not decompress the user-data because the header is missing (The API does not set 'Content-Encoding' = 'gzip'). ISSUE TYPE Bugfix Pull Request COMPONENT NAME ec2_metadata_facts ADDITIONAL INFORMATION Reviewed-by: Mike Graves <mgraves@redhat.com> Reviewed-by: Mark Chappell Reviewed-by: Alina Buzachis (cherry picked from commit f1fe603)

… is compressed (#1575) (#1580) [PR #1575/f1fe6030 backport][stable-6] ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed This is a backport of PR #1575 as merged into main (f1fe603). SUMMARY Handle decompression when user-data is compressed. The fetch_url method from ansible.module_utils.urls does not decompress the user-data because the header is missing (The API does not set 'Content-Encoding' = 'gzip'). ISSUE TYPE Bugfix Pull Request COMPONENT NAME ec2_metadata_facts ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis

…le-collections#1575) ssm_parameter: add support for tags (ansible-collections#1573) SUMMARY Adding support for tags following community guidelines and other practices from other modules. secretsmanager_secret was used along with helper functions from ec2 code. Addresses open issue for feature request ansible-collections#1573. ISSUE TYPE Feature Pull Request COMPONENT NAME ssm_parameter ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None> Reviewed-by: Michael Haskell (mikehas) <None> Reviewed-by: Dennis Qian <None> Reviewed-by: Markus Bergholz <git@osuv.de> Reviewed-by: Mark Chappell <None>

alinabuzachis changed the title ~~Handle decompression when user-data is compressed.~~ [WIP] Handle decompression when user-data is compressed. May 26, 2023

Handle decompression when user-data is compressed. The fetch_url meth…

8ac8799

…od from ansible.module_utils.urls does not decompress the user-data unless the header explicitly contains 'Content-Encoding' = 'gzip'. Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

alinabuzachis force-pushed the ec2_metadata_facts/user-data-decompression branch from af14a87 to 8ac8799 Compare May 26, 2023 11:42

alinabuzachis changed the title ~~[WIP] Handle decompression when user-data is compressed.~~ ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed May 26, 2023

gravesm reviewed May 26, 2023

View reviewed changes

plugins/modules/ec2_metadata_facts.py Outdated Show resolved Hide resolved

plugins/modules/ec2_metadata_facts.py Outdated Show resolved Hide resolved

alinabuzachis force-pushed the ec2_metadata_facts/user-data-decompression branch 4 times, most recently from c5dacef to 275bcb4 Compare May 26, 2023 12:17

Remove base64 decoding as already done by the API

5c13b0a

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

alinabuzachis force-pushed the ec2_metadata_facts/user-data-decompression branch from 275bcb4 to 5c13b0a Compare May 26, 2023 12:18

gravesm reviewed May 26, 2023

View reviewed changes

Handle decompression and decoding separately

94fa1b7

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

alinabuzachis requested a review from gravesm May 26, 2023 13:15

alinabuzachis added backport-6 PR should be backported to the stable-6 branch backport-5 PR should be backported to the stable-5 branch labels May 26, 2023

gravesm approved these changes May 26, 2023

View reviewed changes

alinabuzachis requested a review from tremble May 30, 2023 08:06

tremble reviewed May 30, 2023

View reviewed changes

plugins/modules/ec2_metadata_facts.py Show resolved Hide resolved

Add warning when decompression fails

82a7a6f

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

alinabuzachis force-pushed the ec2_metadata_facts/user-data-decompression branch from 9bd0523 to 82a7a6f Compare May 30, 2023 10:16

tremble approved these changes May 30, 2023

View reviewed changes

tremble reviewed May 30, 2023

View reviewed changes

changelogs/fragments/20230526-ec2_mertadata_facts-handle_compressed_user_data.yml Outdated Show resolved Hide resolved

Fix changelog

8a9542e

Signed-off-by: Alina Buzachis <abuzachis@redhat.com>

tremble reviewed May 30, 2023

View reviewed changes

changelogs/fragments/20230526-ec2_mertadata_facts-handle_compressed_user_data.yml Outdated Show resolved Hide resolved

force type string in changelog

62a1c2e

alinabuzachis added the mergeit Merge the PR (SoftwareFactory) label May 30, 2023

softwarefactory-project-zuul bot merged commit f1fe603 into ansible-collections:main May 30, 2023
71 checks passed

patchback bot mentioned this pull request May 30, 2023

[PR #1575/f1fe6030 backport][stable-6] ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1580

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1575

ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1575

alinabuzachis commented May 26, 2023 •

edited

Loading

gravesm May 26, 2023

softwarefactory-project-zuul bot commented May 26, 2023

softwarefactory-project-zuul bot commented May 26, 2023

softwarefactory-project-zuul bot commented May 30, 2023

softwarefactory-project-zuul bot commented May 30, 2023

patchback bot commented May 30, 2023 •

edited

Loading

patchback bot commented May 30, 2023 •

edited

Loading

ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1575

ec2_metadata_facts - Handle decompression when EC2 instance user-data is compressed #1575

Conversation

alinabuzachis commented May 26, 2023 • edited Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ADDITIONAL INFORMATION

gravesm May 26, 2023

Choose a reason for hiding this comment

softwarefactory-project-zuul bot commented May 26, 2023

softwarefactory-project-zuul bot commented May 26, 2023

softwarefactory-project-zuul bot commented May 30, 2023

softwarefactory-project-zuul bot commented May 30, 2023

patchback bot commented May 30, 2023 • edited Loading

Backport to stable-5: 💔 cherry-picking failed — conflicts found

Backporting merged PR #1575 into main

patchback bot commented May 30, 2023 • edited Loading

Backport to stable-6: 💚 backport PR created

alinabuzachis commented May 26, 2023 •

edited

Loading

patchback bot commented May 30, 2023 •

edited

Loading

patchback bot commented May 30, 2023 •

edited

Loading