Skip to content

Increase event queue sizes and log about dropped events#235

Merged
OpsBotPrime merged 3 commits intomasterfrom
robbert-vdh/increase-event-queue-sizes
Oct 10, 2023
Merged

Increase event queue sizes and log about dropped events#235
OpsBotPrime merged 3 commits intomasterfrom
robbert-vdh/increase-event-queue-sizes

Conversation

@robbert-vdh
Copy link
Member

This reduces the chance that webhook events are getting lost when Hoff receives a couple dozen of them in one go. The previous queues were bounded to 10 elements. This increases them to 128 elements, and it prints am message to STDOUT if events are dropped so we can set up alerts for that.

CC @bertptrs

@robbert-vdh robbert-vdh requested a review from crtschin October 10, 2023 08:53
Copy link

@crtschin crtschin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, might be a good idea to make it configurable to specify the queue sizes via the config so we don't need to make a PR in case this needs to be adjusted further, can you make an issue?

@robbert-vdh
Copy link
Member Author

Yeah, good idea! I'll get right onto that.

@OpsBotPrime merge

@OpsBotPrime
Copy link

Pull request approved for merge by @robbert-vdh, rebasing now.

Approved-by: robbert-vdh
Auto-deploy: false
@OpsBotPrime
Copy link

Rebased as 5a27c6f, waiting for CI …

@OpsBotPrime
Copy link

CI job 🟡 started.

@OpsBotPrime OpsBotPrime merged commit 5a27c6f into master Oct 10, 2023
@OpsBotPrime OpsBotPrime deleted the robbert-vdh/increase-event-queue-sizes branch October 10, 2023 09:22
Hoff dropping events when receiving dozens of webhook events at the exact
same time.
* When Hoff drops a webhook event because its internal event queue is full, it
will now log the evenet to STDOUT>
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/evenet/event/. Also, is that > supposed to be a .?

@robbert-vdh robbert-vdh changed the title Increase event queue sizes and log about dropepd events Increase event queue sizes and log about dropped events Oct 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants