Harden real-time re-indexing pipeline#65
Merged
Conversation
jpr5
force-pushed
the
blitz/webhook-hardening/integration
branch
from
a2dd616 to
7e87abf
Compare
express.raw() defaulted to 100KB, silently rejecting large GitHub push payloads with 413. Raised to 25mb on all 3 webhook routes. Health monitor commit-drift threshold raised from 6h to 25h to match daily reindex cadence. Added aimock-docs as third monitored instance. Added webhook delivery stats to /health endpoint for observability.
Queue-level dedup prevents redundant jobs for the same repo/source. Concurrent drain replaces serial processing — independent repos now reindex in parallel (configurable max, default 3). Per-repo mutex serializes same-repo jobs. do-while re-check after final wait prevents jobs queued during drain from being missed. Wired webhook_deliveries cleanup into nightly reindex cycle (30-day retention, mirrors query_log pattern).
New webhook_deliveries table records every webhook receipt with source, event type, decision (queued/ignored/error), reason, and payload size. All 3 handlers (GitHub, Slack, Discord) log at every decision point. Fire-and-forget pattern ensures tracking never blocks webhook processing. 30-day retention cleanup wired into nightly reindex cycle.
Added --config flag to specify which deploy YAML to read (defaults to mcp-docs.yaml for backward compat). Fixed PATCH to include events[]=push so existing webhooks get correct event subscriptions. Initialized REPOS array before conditional to prevent unbound variable under set -u.
jpr5
force-pushed
the
blitz/webhook-hardening/integration
branch
from
7e87abf to
2ff2d50
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
express.raw()defaulted to 100KB, now 25mbwebhook_deliveriesDB table logs every webhook receipt with decision/reason; stats exposed on/health--configflag for multi-instance use, PATCH includesevents[]=push, REPOS array initializedWhy
The root cause of spurious "commit drift" health alerts was
express.raw()silently dropping GitHub push payloads larger than 100KB with a 413 — no logs on the Pathfinder side. Webhook-triggered re-indexing was already built but not firing for large pushes. The delivery tracking table ensures this class of silent failure is visible going forward.Test plan
tsc --noEmitclean