Use Exponential Message Delivery Backoff in JetStream dispatcher

[mfaizanse]

Pre-requisites

• JetStream supports Exponential Message Delivery Backoff
• Exponential backoff or msg.NakWithDelay(time.Second * 10) feature is released in https://github.com/nats-io/nats.go

Description

Currently, the JetStream dispatcher will not NAK the msg if it fails to deliver it to the sink, so that it timeouts after AckWait period and then redeliver. It will keep on trying for some time with a constant backoff.

Once Exponential Message Delivery Backoff is supported by JetStream, we should use the exponential backoff in our JetStream dispatcher instead of constant backoff.

• (configurable) maximum redelivery
• (configurable) delivery timeout

Attachments

Related issue: #13625

[vpaskar]

Seems that the backoff functionality was already introduced in the v2.7.1: https://github.com/nats-io/nats-server/releases/tag/v2.7.1,

see the comment in the original NATS issue

[kyma-stale-bot]

This issue has been automatically marked as stale due to the lack of recent activity. It will soon be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically marked as stale due to the lack of recent activity. It will soon be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed due to the lack of activity.

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity.
Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[kyma-bot]

This issue or PR has been automatically marked as stale due to the lack of recent activity.
Thank you for your contributions.

This bot triages issues and PRs according to the following rules:

• After 60d of inactivity, lifecycle/stale is applied
• After 7d of inactivity since lifecycle/stale was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Close this issue or PR with /close

If you think that I work incorrectly, kindly raise an issue with the problem.

/lifecycle stale

[grischperl]

Now that JetStream supports both exponential backoff and NakWithDelay, it was decided to use the latter as the EC will respond with a Nak. The msg is put back in the queue and waits previously configured time (30sec) until the msg is send again (max. 100 retries).

Additionally, in case NakWithDelay fails and no response is sent, AckWait is configured to redeliver every 30sec.
#17102

The maximum redelivery and timeout are not yet configurable.