Skip to content

Release v19.104.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 20 Oct 18:29
v19.104.0
125d668
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Summary

One failing webhook receiver no longer holds up the rest. Webhook observers now have their own retry policy with a 24-hour budget, honour Retry-After, back off per target when a receiver keeps failing, and keep their retry deadlines across restarts.

Added

  • A Webhooks:Retry configuration section, separate from the global observer retry settings. By default a failing partition is retried after 1 s, 5 s, 30 s, 2 min, 10 min and 30 min, then hourly, each with ±20% jitter. After 24 hours it is parked. The webhook observer pauses when 5 partitions are parked or at least half of its failing partitions are parked. BackoffScheduleSeconds, MaximumBackoffDelay, MaxRetryAttempts, RetryBudget and the quarantine thresholds can all be overridden. (#5106)
  • A circuit breaker per webhook target (scheme, host and port). While it is open, delivery is deferred until its deadline without using up a retry attempt. The chronicle-webhook-circuit-opened and chronicle-webhook-circuit-rejected counters are tagged by target authority. (#5106)
  • Webhooks:MaximumRetryAfterSeconds (default 24 hours) caps how long a receiver's Retry-After can defer delivery. (part of #5108)

Changed

  • Webhook retry deadlines, retry counts and the resume position are stored, so a restart or reactivation no longer makes delivery retry early or use up an extra attempt. A manual partition retry overrides a pending deadline. (#5106)
  • A 429 or 503 with a valid Retry-After (seconds or HTTP date) defers the partition until that time without using up a retry attempt. (part of #5108)
  • A 4xx response other than 408, 410 and 429 is treated as a configuration problem: the webhook observer pauses instead of retrying the same request. Fix the configuration and clear the quarantine to resume. (part of #5108)
  • A 410 Gone response parks the partition immediately, with no automatic retries. (part of #5108)
  • Each webhook delivery makes a single HTTP attempt. Retries are handled by the observer instead of retrying inside the HTTP client, so a failing receiver no longer ties up a worker. Webhooks:RetryDelaySeconds is kept for compatibility but no longer has any effect. (#5106)
  • Catch-up and replay jobs wait for a partition's pending retry deadline instead of delivering around it. (#5106)

Fixed

  • A webhook receiver that keeps failing or throttling no longer holds up delivery to other webhook targets. (#5106)
Assets 2
Loading