Overview

This is an example of consuming SQS messages in AWS with the potential of having a very long processing time for each message. It uses the long polling method of retrieving messages. Upon message receipt, it will spin up a "visibility timeout extender" goroutine that will keep extending the message's visibility timeout while being processed.

How to run

$ export AWS_REGION=<some-region>
$ export AWS_ACCESS_KEY_ID=<some-value>
$ export AWS_SECRET_ACCESS_KEY=<some-value>

# build the sample
$ go build -v

Using AWS console, create an SQS queue with testlengthyprocessing as name. The default value for "Default Visibility Timeout" should be 30s. Then run the sample, preferably two instances to confirm that only one running instance will process the message even past the queue's visibility timeout of 30s.

# run the sample
$ ./sqs-lengthy-consume
2018/10/05 12:02:20 start listen, queue=testlengthyprocessing, visibility=30s

Finally, send a test message using AWS console's "Send a Message" option under "Queue Actions" menu.

2018/10/05 12:02:34 raw=test
2018/10/05 12:02:34 simulate lengthy work (1min)
2018/10/05 12:02:54 visibility timeout for [AQEBol+zZ7...] updated to 30
2018/10/05 12:03:14 visibility timeout for [AQEBol+zZ7...] updated to 30
2018/10/05 12:03:34 fn=processCallback, duration=1m0.003743499s
2018/10/05 12:03:34 visibility timeout extender done for [AQEBol+zZ7...]