fix(analytics): record batch evaluate failures in the Decision Audit - #389
Merged
Conversation
routing_evaluate_batch records a request hit on entry, but fail_batch only incremented the Prometheus failure counter -- it never wrote a domain event. Every 4xx from the batch endpoint therefore showed in the audit as an evaluation that started and never concluded, while Hyperswitch logged the failed routing event on its own dashboard. The single endpoint has always recorded both, through fail_preview, so the two endpoints disagreed about what a failure looks like. fail_batch now records the same RoutingEvaluateError event, and each of the five failure sites names its stage. The stages carry a batch_ prefix because AnalyticsRoute has no batch variant -- batch traffic is filed under routing_evaluate, so the stage is what tells the two apart. Splitting the route label is a separate change, since it would move existing traffic between labels. A whole-batch failure has no single entry to attribute, so the event is built from one representative request: the caller, the shared fallback and algorithm_for, and the first entry's parameters. Verified against a live engine: an oversized batch on the current binary records only request_hit, and with this change records request_hit plus routing_evaluate_error/batch_size_validation_failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
A
4xxfrom/routing/evaluate/batchshowed on the Hyperswitch dashboard but not in the Decision Audit.The two evaluate endpoints disagreed about what a failure looks like:
fail_preview(/routing/evaluate)record_error(...)fail_batch(/routing/evaluate/batch)fail_batchincrementedAPI_REQUEST_COUNTERand returned. It never wrote aDomainAnalyticsEvent.Worse than simply missing: the handler does call
record_request_hiton entry, so a failed batch call appeared in the audit as an evaluation that started and never concluded — and it is counted by the call-count metric added in #387 with no matching outcome.Five failure paths were affected: batch too large, routing config unavailable, parameter validation,
resolve_active_algorithm(the batch twin of the no-active-rule 400), and algorithm-data parse.The change
fail_batchnow records the sameRoutingEvaluateErroreventfail_previewdoes, and each site names its stage.Stages carry a
batch_prefix deliberately:AnalyticsRoutehas no batch variant, so batch traffic is filed underrouting_evaluate. Until that is split, the stage is what distinguishes the two. Splitting the route label is intentionally not in this PR — it would move existing batch volume between labels and shift any dashboard filtering onrouting_evaluate.A whole-batch failure has no single entry to attribute, so the event is built from one representative request: the caller, the shared
fallback_outputandalgorithm_for, and the first entry's parameters.Verified against a live engine
Same oversized batch (51 entries, limit 50) fired at both binaries. Both returned an identical
400; only the audit differs:analytics_domain_eventsauditprobe_controlmainrequest_hitonly — the 400 left no traceauditprobe_merchantrequest_hit+routing_evaluate_error/batch_size_validation_failed/failurecargo clippyandcargo fmt --checkclean.Note
The batch endpoint has no Playwright coverage at all — no spec under
tests/exercises/routing/evaluate/batch. Worth adding separately.🤖 Generated with Claude Code
Closes #403