[FR] Add ?strict=true mode to /api/health/ready for empty-shard detection

[Harsh23Kashyap]

Problem

GET /api/health/ready (added in #1506 / PR #1507) probes Postgres, Redis, and Zoekt and returns 503 / status: "degraded" if any check fails. The Zoekt check uses an empty List RPC, which the gRPC server treats as a successful round-trip even when no repositories are indexed. That is the right default — the question a readiness probe answers is "can this instance talk to its dependencies?" — but it leaves a real operational gap: a freshly-restarted Sourcebot that has not yet completed a single sync, or a Sourcebot whose connection sync is silently broken, still reports status: "ok" and gets routed traffic.

Motivation

A self-hosted operator wiring Sourcebot into a Kubernetes readinessProbe (or a load-balancer health check) typically wants two distinct signals:

• Soft ready — the process is up and the dependencies are reachable. The current default.
• Strictly ready — the process is up, the dependencies are reachable, and the instance can actually serve a search that returns results. A node that has not finished indexing anything is not strictly ready, even if all three RPCs succeed.

Today there is no way to ask the readiness probe the second question. A node can pass liveness, pass readiness, and still be useless to a search request.

Proposed solution

Add a ?strict=true query parameter to GET /api/health/ready. When strict=true:

• The Zoekt check inspects the List response. If it is empty (repos === undefined or repos.length === 0), the check is reported as status: "degraded" (or a new status: "empty" value), and the overall response becomes 503 / status: "degraded".
• The Postgres and Redis checks behave identically to the non-strict path.
• The response shape gains a strict: true | false field so callers can confirm which mode was applied.

When strict is absent or strict=false (the default), behavior is unchanged from #1506.

Example

$ curl -s 'http://localhost:3000/api/health/ready?strict=true' | jq
{
  "status": "degraded",
  "strict": true,
  "checks": {
    "postgres": { "status": "ok", "latencyMs": 3 },
    "redis":    { "status": "ok", "latencyMs": 1 },
    "zoekt":    { "status": "degraded", "latencyMs": 18, "error": "no repositories indexed" }
  }
}

Alternatives considered

• Add a separate /api/health/ready/strict endpoint. Rejected: duplicate route, duplicate auth, duplicate PostHog tracking, no real benefit over a query param.
• Make strict the default and add a ?loose=true opt-out. Rejected: a backwards-incompatible behavior change for a recently-merged endpoint is a poor first move; the loose interpretation is the natural one for "can we talk to our dependencies".
• Caching the response and using the cached value to drive the strict signal. Out of scope. Caching is a separate, larger follow-on already noted in [FR] Add /api/health/ready endpoint with dependency health checks (Postgres, Redis, Zoekt) #1506's "future work" section.

Scope

• New query parameter on the existing route.
• Per-check shape gains a strict-relevant error string for the Zoekt case.
• New tests covering: strict with empty shards → 503; strict with shards → 200; non-strict with empty shards → 200; invalid strict value → 400.
• Docs (docs/docs/api-reference/health.mdx) updated with one example showing the strict path and a Kubernetes readinessProbe snippet that uses it.
• No new dependencies, no schema migration, no new files. Pure addition to the existing route.

Acceptance criteria

• GET /api/health/ready?strict=true returns 503 / status: "degraded" when Zoekt has zero indexed repos, and 200 / status: "ok" when Zoekt has at least one.
• GET /api/health/ready (no query param) and ?strict=false keep their current behavior.
• ?strict=invalid returns 400 with a clear error message.
• The response always includes the strict field so operators can confirm the mode.
• New tests cover all four cases above plus the parallel-execution invariant from [FR] Add /api/health/ready endpoint with dependency health checks (Postgres, Redis, Zoekt) #1506.
• Lint, type-check, and full test pass on the modified files.
• Docs updated; CHANGELOG.md entry added under [Unreleased] → Added.

Backward compatibility

Fully backwards compatible. The default behavior is unchanged. Operators who do not pass strict=true see the same response as today.

Risks

• The Zoekt List response shape differs between proto versions. Need to handle both repos (RepoListFieldRepos) and reposMap (RepoListFieldReposMap) — empty in both cases means strict-not-ready.
• Operators using a default ?strict=true probe who do not see any indexed repos in their dev environment may be confused why readiness fails. Mitigated by the strict field in the response and a clear error string on the Zoekt check.

Related

• Closes the "strict=true mode" follow-on item in the "Out of scope" section of [FR] Add /api/health/ready endpoint with dependency health checks (Postgres, Redis, Zoekt) #1506.
• Builds on GET /api/health/ready from [FR] Add /api/health/ready endpoint with dependency health checks (Postgres, Redis, Zoekt) #1506 / PR feat(web): add /api/health/ready endpoint with dependency health checks #1507.