EC2MetadataUtils is very slow after 1.11.678

[kliakos]

On 1.11.678 release, the EC2CredentialsUtils was replaced with InstanceMetadataServiceResourceFetcher.

A call to EC2MetadataUtils.getInstanceInfo() which used to last a few milliseconds, now takes a few seconds! On 1.11.678 it takes 10 seconds to respond, while on 1.11.749 it is quite better, but still it takes 2 seconds.

Is this something related to Metadata Service Version 2, or the SDK itself? If this is by design the way it works and the latency issue can not be fixed, we should at least be given the option to use the v1 service via the SDK.

[kliakos]

Moreover, nothing was mentioned about this modification on the 1.11.678 release notes.

https://github.com/aws/aws-sdk-java/blob/master/CHANGELOG.md#111678-2019-11-19

[kliakos]

Tried aws-sdk-java-v2, but with no luck. Still the EC2MetadataUtils.getInstanceInfo() takes a second at least.

[debora-ito]

Hi @kliakos yes, this is related to the Instance Metadata Service V2 change, we have multiple reports of increased latency when calling the new service. We've opened an internal ticket to the EC2 team behind the Metadata Service and they are investigating.

Not mentioning this change in the CHANGELOG was a miss in our part.

I'll update here when we hear back from the EC2 team.

[ffeltrinelli]

Hi! Any updates on this one? I'm also having intermittent latency issues (sometimes reaching the non-configurable 1sec read timeout) while the SDK is fetching credentials from IMDS.

[kliakos]

Has there been any improvement with this issue? I tried 1.12.189 and the latency seems to be around the area of ~1sec.

[ffeltrinelli]

Hi! I talked about this issue and described our custom solution in this article.

[kliakos]

Hi! I talked about this issue and described our custom solution in this article.

EC2MetadataUtils is just a getter-only util class with static methods. It doesn't accept an AWSCredentialsProvider, thus the solution of overriding it is not possible.

[debora-ito]

There's no recent reports of IMDSv2 high latency, so I'll go ahead and close this.

If you are still seeing intermittent latency issues, please test a more recent version of the SDK 1.12.x, and then open a new fresh issue with repro steps.

[github-actions]

This issue is now closed.

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.