Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry for all tag fetching #3765

Merged
merged 3 commits into from Apr 1, 2024
Merged

Retry for all tag fetching #3765

merged 3 commits into from Apr 1, 2024

Conversation

t0yv0
Copy link
Member

@t0yv0 t0yv0 commented Apr 1, 2024

TestRandomTagsComabinationsGo periodically flakes up on failing to retrieve tags from AWS. This small change should add a retry to hopefully make it more robust.

Fixes #3403

@t0yv0 t0yv0 requested a review from mjeffryes April 1, 2024 15:10
@t0yv0 t0yv0 mentioned this pull request Apr 1, 2024
Copy link

github-actions bot commented Apr 1, 2024

Does the PR have any schema changes?

Does the PR have any schema changes?

Looking good! No breaking changes found.
No new resources/functions.

Maintainer note: consult the runbook for dealing with any breaking changes.

Copy link

github-actions bot commented Apr 1, 2024

Is README.md missing any configuration options?

assumeRoleWithWebIdentity not found in Configuration section
customCaBundle not found in Configuration section
defaultTags not found in Configuration section
ec2MetadataServiceEndpoint not found in Configuration section
ec2MetadataServiceEndpointMode not found in Configuration section
httpProxy not found in Configuration section
httpsProxy not found in Configuration section
ignoreTags not found in Configuration section
noProxy not found in Configuration section
retryMode not found in Configuration section
s3UsEast1RegionalEndpoint not found in Configuration section
sharedConfigFiles not found in Configuration section
skipRequestingAccountId not found in Configuration section
stsRegion not found in Configuration section
tokenBucketRateLimiterCapacity not found in Configuration section
useDualstackEndpoint not found in Configuration section

Please add a description for each of these options to README.md. Details about them can be found in either the upstream docs or schema.json.

Comment on lines 222 to 223
retries := 5
client := s3sdk.New(sess, &aws.Config{MaxRetries: &retries})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to leave a comment here noting the reason we need to retry

@@ -242,7 +244,8 @@ func fetchAppConfigTags(t *testing.T, arn string) map[string]string {
sess := session.Must(session.NewSessionWithOptions(session.Options{
SharedConfigState: session.SharedConfigEnable,
}))
client := appconfigsdk.New(sess)
retries := 5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@t0yv0 t0yv0 force-pushed the t0yv0/retry-flaky-tags-check branch from 37c1377 to 71cbd64 Compare April 1, 2024 20:38
@t0yv0
Copy link
Member Author

t0yv0 commented Apr 1, 2024

Caught a few more different reasons for this flaky test; this is really difficult to reproduce but I'm guessing it may be about the eventual consistency of having actual tags in AWS. Add some outer retries around fetching actual tags as well. Let's see if this stops flaking. Going to close it optimistically.

@t0yv0 t0yv0 enabled auto-merge (squash) April 1, 2024 20:40
@t0yv0 t0yv0 merged commit 0e14dc9 into master Apr 1, 2024
23 checks passed
@t0yv0 t0yv0 deleted the t0yv0/retry-flaky-tags-check branch April 1, 2024 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TestRandomTagsCombinations sporadic test failures
2 participants