Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

S3Client deleteObjects with empty object list throws NPE #4021

Closed
joernschumacher0001 opened this issue May 17, 2023 · 5 comments
Closed

S3Client deleteObjects with empty object list throws NPE #4021

joernschumacher0001 opened this issue May 17, 2023 · 5 comments
Assignees
@debora-ito
Labels
bug This issue is a bug.

Comments

@joernschumacher0001
Copy link

joernschumacher0001 commented May 17, 2023 •
edited

Describe the bug

Passing a DeleteObjectsRequest with an empty object List to the default s3 client throws a NullPointerException.

Expected Behavior

Return without an error.

Current Behavior

software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 500, Request ID: null)
        at app//software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
        at app//software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108)
        at app//software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:85)
        at app//software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:43)
        at app//software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:95)
        at app//software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:2[70](https://github.com/otto-ec/dv_opal-events/actions/runs/4979035935/jobs/8910159416#step:4:71))
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)
        at app//software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
        at app//software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at app//software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
        at app//software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)
        at app//software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at app//software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
        at app//software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
        at app//software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)
        at app//software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
        at app//software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:1[71](https://github.com/otto-ec/dv_opal-events/actions/runs/4979035935/jobs/8910159416#step:4:72))
        at app//software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:82)
        at app//software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:179)
        at app//software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:76)
        at app//software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
        at app//software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:56)
        at app//software.amazon.awssdk.services.s3.DefaultS3Client.deleteObjects(DefaultS3Client.java:2[73]
        at app//our.service.aws.S3Service.deleteObjects(S3Service.java:1[75]

This does not happen if we verify the list we pass to deleteObjects() is not empty.

Reproduction Steps

final var awsCredentialProvider = ...;
final var s3Bucket = ...;
final var s3Client = S3Client.builder()
            .credentialsProvider(awsCredentialsProvider)
            .build();

        final var deleteRequest = DeleteObjectsRequest.builder()
            .delete(builder -> builder.objects(Collections.emptyList()))
            .bucket(s3Bucket)
            .build();
        final var response = s3Client.deleteObjects(deleteRequest);

This is a shortcut; in our case, we retrieved a list via listObjects() that was empty. We were not aware that we had to check it wasn't before passing it to the delete request.

Possible Solution

Check early if the list is empty.
If this behavior is desired on your side, throw an exception right then to fail fast. This could be done as early as in the DeleteObjectsRequestBuilder#build()
If not, return success immediately.

Additional Information/Context

No response

AWS Java SDK version used

2.20.47

JDK version used

Java 17.0.7.7.1 (Corretto) from https://corretto.aws/downloads/resources/17.0.7.7.1/amazon-corretto-17.0.7.7.1-linux-x64.tar.gz

Operating System and version

local Windows 10 and github action - not sure of OS/version there.
@joernschumacher0001 joernschumacher0001 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels May 17, 2023
@debora-ito
Copy link
Member

debora-ito commented May 17, 2023 •
edited

@joernschumacher0001 that's not really a NullPointerException, it's a 500 error response with no message. The SDK tried to parse the error message but showed null since the service response had no message.

As a general rule, the Java SDK doesn't do client-side validation of attributes. This avoids issues in case the service changes a validation rule later in time and customer are still using an old version of the SDK.

I tried to reproduce the error, but in my case I got a 400 Bad Request with :
software.amazon.awssdk.services.s3.model.S3Exception: The XML you provided was not well-formed or did not validate against our published schema (Service: S3, Status Code: 400, Request ID: xxx, Extended Request ID: xxx)

Are you using an S3 endpoint, or a third-party endpoint?

@debora-ito debora-ito removed the needs-triage This issue or PR still needs to be triaged. label May 17, 2023
@debora-ito debora-ito self-assigned this May 17, 2023
@debora-ito debora-ito added the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. label May 18, 2023
@github-actions
Copy link

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.

@github-actions github-actions bot added the closing-soon This issue will close in 4 days unless further comments are made. label May 22, 2023
@joernschumacher0001
Copy link
Author

@debora-ito You're right, it's not a NullPointerException but a failure response from the server, my bad.
I'm not sure what you mean by "an S3 endpoint", I'm creating an S3Client with a bucket name with the code I posted.

I see why it would be a general rule to not do client-side validation and guess I have to (somewhat grumbingly) accept that decision.

@github-actions github-actions bot removed closing-soon This issue will close in 4 days unless further comments are made. response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. labels May 24, 2023
@debora-ito
Copy link
Member

Let us know if you have any other question.

@github-actions
Copy link

github-actions bot commented Jun 6, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@debora-ito debora-ito

Labels
bug This issue is a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@debora-ito @joernschumacher0001