Skip to content

PUT /v1/buy/paymentInfos: 27.5% of calls over 1 s, p95 at 10 s #4525

Description

@TaprootFreak

PUT /v1/buy/paymentInfos is slow for a large share of calls. This is a customer-facing path — it is what a user waits on when starting a purchase.

Measurement

Production logs, 24 h window ending 2026-07-30 22:45 CEST, 3,088 calls:

Percentile Latency
median 181 ms
p75 1,309 ms
p90 5,190 ms
p95 9,962 ms
p99 18,157 ms
max 56,746 ms
Threshold Calls Share
> 1 s 848 27.5%
> 5 s 317 10.3%
> 10 s 153 5.0%

So roughly every fourth call takes more than a second, and one in twenty takes more than ten.

This is not an error path

Status codes over the same window: 2,786×200, 275×400, 14×401, 10×409, 2×403, 1×500.

The slow calls are overwhelmingly successful ones — of the fifteen slowest, thirteen returned 200, including the 56.7 s outlier. So this is not a timeout-then-fail pattern; users are served correctly, just very late.

Spread over the day

The slowest calls are scattered across the whole 24 h (09:05, 12:09, 14:51, 18:11, 21:02, 23:39, 02:22 …) rather than clustering in one window, so it does not look tied to a single daily peak.

Scope

Not related to the performance work merged today (#4519, #4520, #4521) — those touched the financial dashboard and the ledger jobs, not this route. It surfaced while measuring the effect of those changes across all endpoints, where this route stood out as the slowest remaining one by a wide margin.

Cause is not investigated here — this issue documents the timing only.

Reproducing the measurement

Loki, production API stream, over the desired window:

{compose_project="dfxprd-dfx-api", compose_service="dfx-api"} |~ "buy/paymentInfos"

Log lines carry the latency directly, e.g. PUT /v1/buy/paymentInfos 200 14010.284 ms. Note the value is a decimal — a pattern matching only digits before ms silently captures the fractional part instead.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions