Problem deleting message from SQS FIFO queue: The receipt handle has expired

[ggordan]

Hi,

I've recently switched to a FIFO queue after using a standard queue for a while and have encountered an error where I am unable to delete messages from the queue. The message is received, as is the ReceiptHandle however when trying to delete, I get the error above.

Full Error message: Value {VALUE} for parameter ReceiptHandle is invalid. Reason: The receipt handle has expired.

SDK Version: 2.4.17 (also tried on 2.7.18)

Code

  sqs.receiveMessage({
    QueueUrl
  }, (err, payload) => {
    if (err) throw err
    payload.Messages.forEach(message => {
      console.log(`received message ${message.MessageId}`, JSON.parse(message.Body))
      sqs.deleteMessage({ QueueUrl, ReceiptHandle: message.ReceiptHandle }, (err, resp) => {
        if (err) throw err
        console.log('message deleted')
      })
    })
  })
}

Stack trace

| /app/node_modules/aws-sdk/lib/request.js:31
|             throw err;
|             ^
| 
| InvalidParameterValue: Value {VALUE} for parameter ReceiptHandle is invalid. Reason: The receipt handle has expired.
|     at Request.extractError (/app/node_modules/aws-sdk/lib/protocol/query.js:40:29)
|     at Request.callListeners (/app/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
|     at Request.emit (/app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
|     at Request.emit (/app/node_modules/aws-sdk/lib/request.js:668:14)
|     at Request.transition (/app/node_modules/aws-sdk/lib/request.js:22:10)
|     at AcceptorStateMachine.runTo (/app/node_modules/aws-sdk/lib/state_machine.js:14:12)
|     at /app/node_modules/aws-sdk/lib/state_machine.js:26:10
|     at Request.<anonymous> (/app/node_modules/aws-sdk/lib/request.js:38:9)
|     at Request.<anonymous> (/app/node_modules/aws-sdk/lib/request.js:670:12)
|     at Request.callListeners (/app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
|     at Request.emit (/app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
|     at Request.emit (/app/node_modules/aws-sdk/lib/request.js:668:14)
|     at Request.transition (/app/node_modules/aws-sdk/lib/request.js:22:10)
|     at AcceptorStateMachine.runTo (/app/node_modules/aws-sdk/lib/state_machine.js:14:12)
|     at /app/node_modules/aws-sdk/lib/state_machine.js:26:10
|     at Request.<anonymous> (/app/node_modules/aws-sdk/lib/request.js:38:9)
|     at Request.<anonymous> (/app/node_modules/aws-sdk/lib/request.js:670:12)
|     at Request.callListeners (/app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
|     at callNextListener (/app/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
|     at IncomingMessage.onEnd (/app/node_modules/aws-sdk/lib/event_listeners.js:231:11)
|     at emitNone (events.js:91:20)
|     at IncomingMessage.emit (events.js:185:7)

Queue Details

Default Visibility Timeout: 0 seconds
Message Retention Period: 1 days
Receive Message Wait Time: 0 seconds
Messages Available (Visible): 107
Delivery Delay: 0 seconds
Content-Based Deduplication: Enabled

Thanks,
Gordan

[ggordan]

So it looks like you can work around this by calling deleteMessageBatch which appears to delete the message successfully (the callback does not return an error). However, the message continues to be sent up to the Maximum Receives count, at which point it is moved to the dead letter queue.

[jeskew]

This is the expected behavior of SQS given a visibility timeout of 0 seconds.

When working with a FIFO queue, DeleteMessage operations will fail if the request is received outside of the visibility timeout window. (This was mentioned in the blog post introducing FIFO queues but does not appear to be restated in the SQS docs.) If the visibility timeout is 0 seconds, the message must be deleted within the same millisecond it was sent or it will be considered to have been abandoned. This can cause SQS to include duplicate messages in the same response to a ReceiveMessage operation if the MaxNumberOfMessages parameter is greater than one, and the SQS team therefore recommends against using a visibility timeout window of 0.

If you use DeleteMessageBatch operations instead, the messages will still fail to delete, which is why they end up in the dead letter queue. DeleteMessageBatch operations report the result of individual delete actions in the body of the response rather than in the response code, so the code snippet above would not throw if all messages in the batch failed to delete.

Closing, as this is not an issue with the SDK. I'll forward this along to the SQS team as feedback on their documentation, as the intended behavior of the service with a zero second visibility timeout window is not clear. Please feel free to reopen if you have any questions or concerns.

[ggordan]

Hi @jeskew . I unfortunately missed the blog post so I wasn't aware. That fixed my issue, thanks!

[route]

@jeskew I have a visibility timeout 30s and same error which can only mean I tried to delete a message after 30s am I right?

[jeskew]

@route Yes, DeleteMessage will fail with an error if the visibility window has already expired.

[taju20910]

Team,
I have set visibility timeout as 90 seconds and I am trying to delete the message within this time range only. Still message in SQS is not getting deleted most of the times . Also when I call message_obj.delete() func it kind of gets stuck there for sometime ( I have a log next which is not getting printed).
Can this in anyway be related to network issue ?
Or is there anything else I am missing out on .

[laisiulong]

Hi team, I am experiencing the same issue as well even changing visibility timeout in codes to non zero . This does not often happen but I still can not fully control of message deletion.

Is there a solution for this?
Thank you
Long

[dwrz]

I had this issue occur on a queue with the default visibiblity timeout of 30 seconds. The receipt handle was reported as expired even though there were still ~29 seconds left on the visibility timeout.

[laisiulong]

As mentioned in aws document, I have to accept failed deletion and handle duplicate messages at consumer end

[HoraceShmorace]

When working with a FIFO queue, DeleteMessage operations will fail if the request is received outside of the visibility timeout window. (This was mentioned in the blog post introducing FIFO queues but does not appear to be restated in the SQS docs.) If the visibility timeout is 0 seconds, the message must be deleted within the same millisecond it was sent or it will be considered to have been abandoned.

Not only is "the message must be deleted within the same millisecond" hilarious, but also this is still nowhere in the SQS documentation.

[payitforwardnow]

Im having the same issue. Timeout is 5 min and yet attempting to delete messages within a few seconds using the Console fails on all attempts.

[gavrilovmilos]

I am facing the same problem, have created new queue with Default Visibility Timeout set to 2 hours and cannot delete message. Always the same error message: "Reason: The receipt handle has expired"

[steadweb]

Purging seems to the be a solution here, if you're certain you can remove all messages from said queue. Otherwise, do not purge the queue if you cannot safely assume you want to remove all messages.

[ccornici]

Same issue here, 1 hour default visibility and tried deleting a test message after 2 min, same error again and again...
Have a similarly configured normal Q and it doesn't happen, only with the FIFO one, AMZN, please fix this.

EDIT: THIS ONLY HAPPENS IN THE AWS UI WHEN MANUally deleting messages,

[knakra92]

@jeskew It worked for me. Perfect!!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.