feat: admin DLQ tooling, observability metrics, and standardized error responses

[gotethry]

What

Implement advanced DLQ admin tooling with filtering and replay, add rate-limit and auth-failure observability metrics, establish standardized error response contract, and add critical database regression tests.

Issues Resolved

Closes #189
Closes #192
Closes #193
Closes #196

Changes

#189 - Admin tooling for DLQ triage and replay operations

• Enhanced GET /api/admin/dlq/inspect with advanced filtering by eventType, retryCount range, and time range
• Added pagination support with offset and limit parameters
• Implemented POST /api/admin/dlq/replay endpoint for selective event replay with safety checks
• Replay operations only target PENDING and RETRIED events to prevent infinite loops from RESOLVED events
• Enhanced audit logging for all DLQ operations with detailed filter metadata

#192 - Rate-limit and auth-failure observability panels

• Added three new Prometheus metrics: rate_limit_hits_total, auth_failures_total, rate_limit_active_violations
• Instrumented all five rate limiters (global, auth, admin, webhook, internal) to record hits by route group
• Added auth failure tracking in adminAuth middleware for missing_token, invalid_token, and auth_error scenarios
• Metrics track by limiter type and endpoint for granular observability and alerting

#196 - Standardized error-response contract across API routes

• Created errorResponse.ts utility module with canonical error format
• All errors now follow: { error: { code, message, details? }, requestId, timestamp }
• Updated error handler middleware to use standardized error responses
• Defined ErrorCodes enum for consistent error categorization across routes
• Helper functions for all common HTTP error codes (400, 401, 403, 404, 409, 429, 422, 500, 503)

#193 - Regression tests for critical Prisma-backed flows

• Added comprehensive test suite in tests/regression.test.ts
• Session creation/cleanup flow tests with expiration validation
• Event persistence and cursor update tests with unique constraint validation
• DLQ write/read/retry flows with status transitions (PENDING -> RETRIED -> RESOLVED)
• Admin audit log persistence and query tests
• All tests verify database integrity and ensure query correctness

[drips-wave]

@gotethry Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits