Skip to content

Conversation

@uds5501
Copy link
Contributor

@uds5501 uds5501 commented Nov 6, 2025

Supports AWS S3's InternalError retries, for more information about the error, please refer - https://repost.aws/knowledge-center/s3-resolve-200-internalerror

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • a release note entry in the PR description.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

@uds5501 uds5501 changed the title Support InternalError code retries Support AWS InternalError code retries Nov 6, 2025
return true;
} else if (e instanceof AmazonS3Exception && e.getMessage().contains("InternalError")) {
// This can happen sometimes when AWS returns a 200 response with internal error message
// https://repost.aws/knowledge-center/s3-resolve-200-internalerror
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link also suggests SlowDown keyword might need similar behaviour, shall we add that too?

Quoting from the link:

A 200 response with InternalError or SlowDown is similar to a 5xx error. Because Amazon S3 is a distributed system, it's normal to see a small percentage of 200 internal errors. It's a best practice to retry these requests.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also check the status code ? Internal error seems to be a bit too loose no ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add both

Copy link
Contributor

@Akshat-Jain Akshat-Jain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kfaraz
Copy link
Contributor

kfaraz commented Nov 6, 2025

Merging since failure is due to an unrelated flaky test.

@kfaraz kfaraz merged commit e38d293 into apache:master Nov 6, 2025
62 of 64 checks passed
@kgyrtkirk kgyrtkirk added this to the 36.0.0 milestone Jan 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants