Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stabilize ec2_vpc_route_table #35975

Merged
merged 3 commits into from
Feb 9, 2018

Conversation

s-hertel
Copy link
Contributor

@s-hertel s-hertel commented Feb 9, 2018

Wait for route table to be present before attempting to use it

Sleep before getting the final state of the route table in case modifications are incomplete

SUMMARY

ec2_vpc_route_table tests frequently fail in CI. This is an attempt to improve that. Running the tests locally, I experience various sporadic failures around 20-30% of the time. With this patch I have run the tests 20 times consecutively without failure.

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

lib/ansible/modules/cloud/amazon/ec2_vpc_route_table.py

ANSIBLE VERSION
2.6.0

Wait for route table to be present before attempting to use it

Sleep before getting the final state of the route table in case modifications are incomplete
@ansibot
Copy link
Contributor

ansibot commented Feb 9, 2018

@ansibot ansibot added aws bugfix_pull_request cloud committer_review In order to be merged, this PR must follow the certified review workflow. module This issue/PR relates to a module. needs_triage Needs a first human triage before being processed. support:certified This issue/PR relates to certified code. labels Feb 9, 2018
@s-hertel
Copy link
Contributor Author

s-hertel commented Feb 9, 2018

Don't merge, looking at the alternative of creating botocore waiters for this instead.

I'll add that for 2.6, but this is an immediate solution.

@s-hertel s-hertel removed the needs_triage Needs a first human triage before being processed. label Feb 9, 2018
@@ -678,6 +685,8 @@ def ensure_route_table_present(connection, module):
purge_subnets=purge_subnets)
changed = changed or result['changed']

# pause to allow route table routes/subnets/associations to be updated before exiting with final state
sleep(5)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of a static sleep here, can you make it conditional on there being a change that should be waited for? Otherwise the happy path of this module (where no changes were needed) will take longer.

@ryansb ryansb merged commit fd33dc6 into ansible:devel Feb 9, 2018
@ansibot ansibot added bug This issue/PR relates to a bug. and removed bugfix_pull_request labels Mar 6, 2018
s-hertel added a commit to s-hertel/ansible that referenced this pull request Apr 27, 2018
…ible#35975)

* Stabilize ec2_vpc_route_table

Wait for route table to be present before attempting to use it

Sleep before getting the final state of the route table in case modifications are incomplete

* Conditionally wait if changes were made

* Simplify logic
s-hertel added a commit to s-hertel/ansible that referenced this pull request Apr 27, 2018
…ible#35975)

* Stabilize ec2_vpc_route_table

Wait for route table to be present before attempting to use it

Sleep before getting the final state of the route table in case modifications are incomplete

* Conditionally wait if changes were made

* Simplify logic

(cherry picked from commit 8fb31ac)
s-hertel added a commit that referenced this pull request May 15, 2018
…9171/38960 (#39440)

* [cloud] Make ec2_vpc_route_table wait for the route to propagate (#35975)

* Stabilize ec2_vpc_route_table

Wait for route table to be present before attempting to use it

Sleep before getting the final state of the route table in case modifications are incomplete

* Conditionally wait if changes were made

* Simplify logic

(cherry picked from commit 8fb31ac)

* Route custom waiter (#36922)

This creates a way for us to use boto3's data-driven waiter support to use custom waiters where Boto3 hasn't implemented them yet.

The only waiter implemented so far is for VPC Route Tables to check that they exist, and this replaces some custom retry code.
(cherry picked from commit a40bce2)

* Use NormalizedOperationMethod to catch ClientErrors so the waiter can handle them properly (#37356)

(cherry picked from commit c9e8aca)

* [cloud] Add custom waiters to stabilize ec2_vpc_subnet module - Fixes #36083 (#37534)

* stabilize ec2_vpc_subnet module

* Add waiters for ec2_vpc_subnet

Clean up integration tests

* Reenable CI for stabilized ec2_vpc_subnet tests

* rename waiters

* Use module_json_aws where applicable

Handle WaiterError first if waiting failed

* Fix traceback when tagging with keys/values that look like booleans

* Fix check mode with tags

* Add integration tests for tags that look like booleans and check mode

* Add waiter for deleting subnet

* Sleep a few seconds after using aws command line

(cherry picked from commit ea943e4)

* Fix sporadic errors in ec2_vpc_subnet integration tests (#38473)

(cherry picked from commit 46f13d3)

* [aws] Skip ec2_vpc_subnet waiters for old botocore versions (#39171)

 Fix ec2_vpc_subnet for botocore versions that do not accept the WaiterConfig parameter
(cherry picked from commit 6b91dae)

* [aws] Increase possible wait time for nonmonotonic subnet attributes (#38960)

(cherry picked from commit c4f0107)

* changelog
@ansible ansible locked and limited conversation to collaborators Apr 27, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
aws bug This issue/PR relates to a bug. cloud committer_review In order to be merged, this PR must follow the certified review workflow. module This issue/PR relates to a module. support:certified This issue/PR relates to certified code.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants