Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add wafv2 modules #450

Merged
merged 11 commits into from Apr 21, 2021
Merged

add wafv2 modules #450

merged 11 commits into from Apr 21, 2021

Conversation

markuman
Copy link
Member

@markuman markuman commented Mar 1, 2021

SUMMARY

Add wafv2 modules

ISSUE TYPE
  • New Module Pull Request
COMPONENT NAME
  • wafv2_resources
  • wafv2_resources_info
  • wafv2_rule_group
  • wafv2_rule_group_info
  • wafv2_web_acl
  • wafv2_web_acl_info
ADDITIONAL INFORMATION

The modules share same module_utils/wafv2.py codebase.
Furthermore, the integration test needs an Application Loadbalancer, that needs itself needs a target group, a subnet and a VPC. This is the reason why the integration test is that long.

@ansibullbot
Copy link

@ansibullbot ansibullbot added community_review integration tests/integration module module needs_triage new_module New module new_plugin New plugin plugins plugin (any type) tests tests labels Mar 1, 2021
@markuman
Copy link
Member Author

markuman commented Mar 1, 2021

aws terminator needs rw permissions for wafv2

botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the ListRuleGroups operation: User: arn:aws:sts::966509639900:assumed-role/ansible-core-ci-test-prod/prod=shippable=ansible-collections=community.aws=1704.22 is not authorized to perform: wafv2:ListRuleGroups on resource: arn:aws:wafv2:us-east-1:966509639900:regional/rulegroup/*

@markuman markuman changed the title add wafv2 modules WIP: add wafv2 modules Mar 19, 2021
@ansibullbot ansibullbot added the WIP Work in progress label Mar 19, 2021
@markuman
Copy link
Member Author

Required: ansible/ansible#73975 for pass 2.9 integration tests

@markuman markuman changed the title WIP: add wafv2 modules add wafv2 modules Apr 8, 2021
@ansibullbot ansibullbot added needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR and removed WIP Work in progress community_review needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html labels Apr 8, 2021
@markuman
Copy link
Member Author

markuman commented Apr 9, 2021

Ok, shippable is running to many tests in parallel.

Here the wafv2 test pass
https://app.shippable.com/github/ansible-collections/community.aws/runs/2238/20/console
Afterwards the wafv2_ip_set set failed (already in main) due time exceeded (45min).

I guess someone must kick one single failed test, until all pass.

@jillr jillr self-assigned this Apr 12, 2021
Copy link
Collaborator

@jillr jillr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial review on the first few files to get you some feedback, will continue reviewing tomorrow

plugins/module_utils/wafv2.py Outdated Show resolved Hide resolved
plugins/modules/wafv2_resources.py Outdated Show resolved Hide resolved
plugins/modules/wafv2_resources_info.py Outdated Show resolved Hide resolved
@ansibullbot ansibullbot added community_review and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR labels Apr 13, 2021
@markuman markuman requested a review from jillr April 13, 2021 19:46
Copy link
Collaborator

@jillr jillr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your patience @markuman (and all the new modules!). If possible I'd like to get a second review on this, maybe @tremble or @goneri? Just a few docs nits and test comments left from me.

I'm planning to release 1.5 in the next 1-2 weeks, and I'm happy block that on this one as we're really close.

plugins/modules/wafv2_web_acl.py Outdated Show resolved Hide resolved
plugins/modules/wafv2_web_acl.py Show resolved Hide resolved
plugins/modules/wafv2_web_acl.py Show resolved Hide resolved
plugins/modules/wafv2_web_acl_info.py Show resolved Hide resolved
tests/integration/targets/wafv2/tasks/alb.yml Outdated Show resolved Hide resolved
tests/integration/targets/wafv2/tasks/alb.yml Show resolved Hide resolved
tests/integration/targets/wafv2/tasks/alb.yml Outdated Show resolved Hide resolved
@markuman markuman requested a review from jillr April 16, 2021 09:03
@goneri goneri self-requested a review April 19, 2021 16:49
plugins/module_utils/wafv2.py Outdated Show resolved Hide resolved
plugins/module_utils/wafv2.py Outdated Show resolved Hide resolved
return response


def wafv2_snake_dict_to_camel_dict(a):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May I suggest to add a unit-test for this function?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm basically it is already covered by the integration test.
I've added different rules circumstances that must be covered by those functions and that must not be covered by those functions.

Copy link
Member

@goneri goneri Apr 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I know and this is the reason why I put suggest in bold. PLEASE, Don't force yourself to do it!

I see two reasons to write unit-tests in your case:

  • sometime the behavior of the API change, it's a way to have a copy of the original expect input.
  • unit-tests run way way way way faster than integration tests. And if they fail we can configure the CI to avoid the functional tests. This way we save the valuable developer time and a lot of CI resources.
  • it's also rather easy for a developer to be able to run the unit-tests locally. It's a different story for the functional tests.

Well, that's 3 reasons actually ^^.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For what it's worth you should be able to copy most of the unit-test framework for these transforms from https://github.com/ansible-collections/amazon.aws/blob/main/tests/unit/module_utils/core/test_scrub_none_parameters.py

retval[item] = a.get(item)
return retval


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, unit-test would be great for the others too :-).

Copy link
Member

@goneri goneri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @markuman.

@goneri goneri added the gate label Apr 21, 2021
@ansible-zuul ansible-zuul bot merged commit 2ee7d6f into ansible-collections:main Apr 21, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 15, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 16, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 17, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 19, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 19, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request Jul 19, 2021
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request May 25, 2022
…nsible-collections#450)

Add botocore requirements to s3_bucket ownership control management

SUMMARY
(get|set|delete)_bucket_ownership_controls requires botocore >= 1.18.11
Because we state our minimum supported version of botocore is 1.16.0 we need to explicitly call this requirement for management of bucket ownership controls.
ISSUE TYPE

Bugfix Pull Request

COMPONENT NAME
s3_bucket
ADDITIONAL INFORMATION
fixes: ansible-collections#449

Reviewed-by: Alina Buzachis <None>
Reviewed-by: None <None>
alinabuzachis pushed a commit to alinabuzachis/community.aws that referenced this pull request May 25, 2022
…nsible-collections#404)

Add constraints.txt and requirements.txt for unit/integration tests

SUMMARY
Now that we state that we support specific minimum versions of the AWS SDKs, make sure we base our unit/integration tests against them such that modules need to explicitly test/request newer versions of the SDKs.
ISSUE TYPE

Feature Pull Request

COMPONENT NAME
tests/integration
tests/unit
ADDITIONAL INFORMATION
Once merged into amazon.aws we should merge this into community.aws
Depends-On: ansible-collections#453
Depends-On: ansible-collections#454
Depends-On: ansible-collections#450
Depends-On: ansible-collections#496
See also: ansible/ansible-zuul-jobs#991

Reviewed-by: Jill R <None>
Reviewed-by: None <None>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community_review integration tests/integration module module new_module New module new_plugin New plugin plugins plugin (any type) tests tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants