pubsub: How to extend deadline past 600s?

[purohit]

Is it possible to modify a message's ack deadline multiple times to get a total deadline of > 600s? That is, if I have jobs which are queue-based, prone to failure, and yet take on average 2000s, I'd like to call modifyAckDeadline at appropriate intervals to extend the deadline lest another consumer receive the message during processing.

I see there's an API endpoint for it: https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.subscriptions/modifyAckDeadline, but I'm not sure how to use this library to do it (or if multiple calls to modifyAckDeadline are even allowed; don't see anything in the docs about that). Right now I've set the ReceiveSettings.MaxExtension = 10 * time.Minute, but that only buys me the ten minutes up-front.

Thanks in advance for your help!

[pongad]

@purohit The short version is ReceiveSettings.MaxExtension has you covered.

The slightly longer version: ReceiveSettings.MaxExtension and SubscriptionConfig.AckDeadline are different.

• AckDeadline configures the pubsub server. If you don't ack or deadline-extend a message for this long, the server might redeliver the message.
• MaxExtension configures the client. In the background, the client will automatically call modifyAckDeadline for you until MaxExtension has passed. After the time passes, the client assumes that you "lost" the message and stops extending.

Note that AckDeadline is still important. If the deadline is set too low, the client will need to call modifyAckDeadline often, which might slow you down. If the deadline is too high, the service will take a while to redeliver if your process crashes, etc.

For your use case, I think it's sufficient to set MaxExtension to something a little higher than 2000s.

[purohit]

@pongad Thanks for the thorough response. I'm really glad .MaxExtension has this behavior!

[TrangPham]

Does the python library also have the MaxExtension feature?

[pongad]

@TrangPham I don't think this is a good place to ask for details on Python lib. Can I point you to https://github.com/googleapis/google-cloud-python ? I'm 99.9% sure it does, but possibly in a different name.