Re-enable ec2_vpc_vpn_facts test

[jillr]

Unable to reproduce failure locally, let's see if shippable still fails

SUMMARY

Test re-enabling ec2_vpc_vpn_facts test.

ISSUE TYPE

• Bugfix Pull Request

COMPONENT NAME

ec2_vpc_vpn_facts integration test

ADDITIONAL INFORMATION

Unable to reproduce timeout failures locally in us-east-1 or us-west-2 though nothing has changed in the test or the module. ¯_(ツ)_/¯

[jillr]

@s-hertel @mattclay I've been unable to get these tests to produce any failure locally or in shippable. What do you think about re-enabling them?

[mattclay]

@jillr Go ahead and enable them. If they start failing again we can disable and investigate further.

[jillr]

Fixes: #53185

[s-hertel]

@jillr 👍

Actually closes: #49751. I think the errors in #53185 are a subset of the ones for #49751, but the ec2_vpc_vgw tests themselves are still disabled. If you'd like to re-enable those too, feel free. If you add Fixes #... in your description or commit msg it will close the issue.

Just some background on unstable tests since these are tricky and sporadic and painstaking to actually solve - As you noted, the module code and tests can be unchanged and suddenly work great (or conversely, fail all over the place). In most cases the reason the tests are unstable is because many AWS services (like EC2, IAM, and S3) use an eventual consistency model. Because of this the failures can be pretty unpredictable (I'd loop tests to run 50 times and do that several times when trying to fix these bugs to address all the different possible places for failure). AWS seems to occasionally make changes to the model which have caused many widespread failures in tests that have never had a problem before. ec2_vpc_vgw was one of these - it has been disabled a couple times now through no fault of Ansible. Even if the tests don't fail when you run them x times, adding AWSRetry and waiters between API calls made by the module are good preventative measures and make the module more resilient against AWS making changes to their distributed system that supports the API.

[jillr]

@s-hertel ok cool, that's super useful info - thanks!