Skip to content

perf(lease-read): background lease warm-up to flatten read-only sawtooth (deferred from #549) #556

@bootjp

Description

@bootjp

Issue

Pure-read workloads see the lease expire every LeaseDuration (700 ms) and pay a LinearizableRead for the first read after expiry. With no writes refreshing the lease, the latency profile becomes a sawtooth: one slow read (~ms) per 700 ms, fast reads in between.

Performance reviewer flagged as HIGH on PR #549.

Fix

Piggy-back on the existing RunHLCLeaseRenewal 1 s timer (kv/coordinator.go:172) to also refresh the lease. The leader already calls Propose for HLC ceiling renewal every 1 s, which itself extends the lease via the Dispatch hook — but the HLC renewal path uses a direct engine.Propose call, not Dispatch, so currently does not refresh the lease.

Options:

  1. Route ProposeHLCLease through the lease-extend hook.
  2. Add a separate background goroutine running at LeaseDuration / 2 (350 ms) that calls LeaseRead.

Option 1 is preferred (no extra goroutine, consolidates with existing renewal).

Acceptance

  • Read-only workload at 100 QPS shows < 1% slow reads (down from ~1.4% at 700 ms expiry / 100 QPS).
  • No regression on write-heavy workloads.

References: PR #549 (deferred).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions