Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Client adds a message attribute name to ReceiveMessageRequest for every call #25

Open
dsw2127 opened this issue Nov 27, 2017 · 2 comments
Open

Client adds a message attribute name to ReceiveMessageRequest for every call #25

dsw2127 opened this issue Nov 27, 2017 · 2 comments

Comments

@dsw2127
Copy link

dsw2127 commented Nov 27, 2017

AmazonSQSExtendedClient.receiveMessage(ReceiveMessageRequest) always appends SQSExtendedClientConstants.RESERVED_ATTRIBUTE_NAME to the list of message attribute names of the request passed in. Therefore if the request object is re-used for multiple invocations, this request can grow and grow. Eventually, if allowed to grow over time, SQS will deny the request with a 413 Request Entity Too Large. This seems to happen when the header gets to be somewhere around 1MB.

com.amazonaws.services.sqs.model.AmazonSQSException: null (Service: AmazonSQS; Status Code: 413; Error Code: 413 Request Entity Too Large; Request ID: null)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
        at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
        at com.amazonaws.services.sqs.AmazonSQSClient.doInvoke(AmazonSQSClient.java:1792)
        at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:1768)
        at com.amazonaws.services.sqs.AmazonSQSClient.executeReceiveMessage(AmazonSQSClient.java:1374)
        at com.amazonaws.services.sqs.AmazonSQSClient.receiveMessage(AmazonSQSClient.java:1350)
        at com.amazon.sqs.javamessaging.AmazonSQSExtendedClientBase.receiveMessage(AmazonSQSExtendedClientBase.java:217)
        at com.amazon.sqs.javamessaging.AmazonSQSExtendedClient.receiveMessage(AmazonSQSExtendedClient.java:349)

Example of message attribute names over four invocations:

MessageAttributeNames: [SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize]
MessageAttributeNames: [SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize, SQSLargePayloadSize]
dsw2127 added a commit to dsw2127/amazon-sqs-java-extended-client-lib that referenced this issue Nov 27, 2017
@dsw2127
Fixes awslabs#25: Only add message attribute name if not already there
5ae6e87 
If someone re-uses the ReceiveMessageRequest object for multiple
calls to AmazonSQSExtendedClient, the latter will add the
RESERVED_ATTRIBUTE_NAME String as a message attribute name every
time. This can cause 413 / Request Entity Too Large errors from
AWS SQS over time.
@dsw2127
Copy link
Author

dsw2127 commented Nov 27, 2017 •
edited
Loading

Workaround is to not re-use (i.e. create a new) ReceiveMessageRequest object for every invocation.

dsw2127 added a commit to dsw2127/amazon-sqs-java-extended-client-lib that referenced this issue Nov 27, 2017
@dsw2127
Fixes awslabs#25: Only add message attribute name if not already there
81ff1f8 
If someone re-uses the ReceiveMessageRequest object for multiple
calls to AmazonSQSExtendedClient, the latter will add the
RESERVED_ATTRIBUTE_NAME String as a message attribute name every
time. This can cause 413 / Request Entity Too Large errors from
AWS SQS over time.
dsw2127 added a commit to dsw2127/amazon-sqs-java-extended-client-lib that referenced this issue Nov 27, 2017
@dsw2127
Fixes awslabs#25: Only add message attribute name if not already there
edd84df 
If someone re-uses the ReceiveMessageRequest object for multiple
calls to AmazonSQSExtendedClient, the latter will add the
RESERVED_ATTRIBUTE_NAME String as a message attribute name every
time. This can cause 413 / Request Entity Too Large errors from
AWS SQS over time.
@robin-aws
Copy link

robin-aws commented Sep 12, 2018 •
edited
Loading

Hi there! Thanks for contributing this fix. We have pulled it into the latest release: https://github.com/awslabs/amazon-sqs-java-extended-client-lib/releases/tag/1.0.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@dsw2127 @robin-aws