Purge cached credentials 15 minutes prior to them expiring #52
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Funnily enough, this (almost) same issue in lyft/metadataproxy is what led us to take a look at go-metadataproxy. :)
Once IAM credentials are 15 minutes away from expiring, the java sdk requests new ones:
https://github.com/aws/aws-sdk-java/blob/1.11.546/aws-java-sdk-core/src/main/java/com/amazonaws/auth/EC2CredentialsFetcher.java#L49-L53
The lyft/metadataproxy would cache credentials until 5 minutes before they expired, which meant that once credentials reached 45 minutes of age, the java sdk would storm it with requests for 10 minutes until they were 55 minutes old, and lyft/metadataproxy would refresh them. The java sdk would then stop, satisfied.
The go-metadataproxy also caches credentials, but, typically for 59 minutes:
This means we'll see 14 minutes of pain instead of 10! Other sdks seem to set a lower limit (ruby sets its threshold at 5 minutes for example, making it a model citizen) so this problem may not show up in all environments.
With this PR I'm proposing setting a threshold of 15 minutes to expiry. If it's preferred though, I can rework this to make the expiration threshold a configurable environment variable, rather than a hardcoded value.