feat(server): add TenantRegistry with per-tenant health tracking

[bokelley]

Closes #619

Adds TenantRegistry — a higher-level multi-tenant primitive that closes the JS↔Python parity gap on createTenantRegistry. Adopters pre-build per-tenant DecisioningPlatform instances and register them; the registry tracks health state and surfaces a synchronous resolve_by_host for the hot request path.

Summary

• New src/adcp/server/tenant_registry.py exporting TenantRegistry, TenantResolution, TenantHealthState, TenantValidator
• TenantRegistry manages four health states per tenant: pending → healthy / disabled, healthy → unverified (graceful-degrade on failed recheck), unverified / disabled → healthy (successful recheck)
• Per-tenant asyncio.Lock prevents TOCTOU races on concurrent recheck() calls; post-validator guard prevents zombie _health entries when unregister() races with an in-flight recheck()
• resolve_by_host is synchronous (in-memory dict updated eagerly by register/unregister) — intentional departure from the JS async variant; the Python registry owns its mapping directly
• validator=None is principal-token mode (always healthy); JWKS or custom health-check adopters pass a (tenant_id, agent_url) -> bool callable (sync or async)
• auto_validate kwarg from the issue proposal dropped per DX review; validator presence is the opt-in

Nits noted (not fixed in this PR):

• _normalize_host does not handle bare bracketless IPv6 hosts (edge case, DNS names only in practice)
• health() returning None for unknown tenants vs platform_for_tenant raising KeyError — intentional for a probe/observation method; documented in docstring

What-tested

• ruff check — clean
• mypy src/adcp/server/tenant_registry.py --ignore-missing-imports — no errors in new file
• pytest tests/test_tenant_registry.py -v — 25/25 passed
  • Lifecycle: register, unregister, resolve_by_host
  • Health state machine: all 4 states, all transition arms
  • Host normalization: port stripping, case folding, URL change
  • Validators: sync, async, raising
  • Concurrency: concurrent rechecks (lock serializes), unregister-during-recheck (no zombie entry)

Pre-PR review

• code-reviewer: approved — ruff F401/I001 nits (both fixed before commit); zombie health entry blocker (fixed); _platforms type tightened to dict[str, DecisioningPlatform]; serve_options returns copy (fixed)
• dx-expert: approved (1 nit noted) — serve_options public property added; docstring usage snippet added

Triage-managed PR. This bot does not currently iterate on
review comments or PR conversation threads (only on the source
issue). To unblock:

• Push fixup commits directly: gh pr checkout <num> →
  fix → push.
• Or re-trigger: comment /triage execute on the source
  issue.

See adcp#3121
for context.

Session: https://claude.ai/code/session_019ucMgF6YS9X5bygYUADztx

---

Generated by Claude Code

[bokelley]

Adopter feedback (filed #619 — most-advanced Python multi-tenant adopter)

Read the full diff. The health-state machine, validator hook, and runtime mutation surface match what we asked for and look great. One critical design issue blocks adoption for us as written: the registration model is eager, and we need lazy.

Eager-vs-lazy mismatch

The PR's docstring example:

```python
for tenant in load_tenants_from_db():
await registry.register(
tenant.id,
agent_url=tenant.agent_url,
platform=build_platform_for(tenant), # ← built at boot
)
```

We currently use `LazyPlatformRouter` precisely because we don't want to pay per-tenant platform-build cost at boot. `build_platform_for_tenant` for a GAM tenant means a network handshake to the GAM auth server, credential fetch from KMS, and inventory-manager construction. Multiplied across N tenants, boot time scales linearly with tenant count. We do this lazily on first request per tenant (core/main.py:191).

If we adopt `TenantRegistry` as written, we'd either:

1. Pay the boot-time cost (regression), or
2. Skip `register()` and lazily call it from a `resolve_by_host` shim — defeating the point of the registry, since unregistered tenants 503

Proposed fix: lazy-factory variant

Add a `register_lazy()` (or `factory=` kwarg on `register`) that takes a builder callable instead of an already-built platform:

```python
async def register_lazy(
self,
tenant_id: str,
*,
agent_url: str,
factory: Callable[[str], Awaitable[DecisioningPlatform]],
await_first_validation: bool = False,
) -> None:
"""Register a tenant with a lazy platform factory.

The platform is built on first resolve_by_host hit, then cached.
Subsequent resolves return the cached instance. Suitable for
deployments with many tenants where eager construction is too
expensive.
"""

```

Internal: the registry holds the factory + a `platform: DecisioningPlatform | None` slot. `resolve_by_host` checks the slot, awaits the factory if empty, fills the slot. The validator + health-state machine work unchanged on the resolved platform.

This composes cleanly with the existing eager API — adopters with few tenants pre-build, adopters with many lazy-build, same registry primitive.

Health states + lazy: behavioral note

With lazy registration, `pending` would mean "registered, factory not yet invoked." The first `resolve_by_host` triggers factory + validator. Health transitions `pending → healthy` on success, `pending → disabled` on factory failure. Matches the eager semantics.

Other observations (non-blocking)

• `_normalize_host` strips port — correct for our setup, but note that some load balancers re-emit non-default ports in `X-Forwarded-Host`. Worth a docstring note.
• The `default_serve_options` plumbing is nice but the docstring example `serve(platform, **registry.serve_options)` only works for single-tenant deployments. Multi-tenant deployments don't pass a single platform to `serve()` — they pass a router. Worth clarifying or removing.
• TOCTOU lock per-tenant is correct. Document that locks aren't garbage-collected on `unregister` (the current code preserves them, which is right — race-resistant — but adopters might wonder).

Bottom line

If the lazy-factory variant lands, we adopt this PR and delete our hand-rolled tenant lifecycle. As written, we'd have to skip it or wrap it. The lazy mode is the difference between "useful primitive for single-host SaaS" and "useful primitive for multi-tenant SaaS at any scale." JS `createTenantRegistry` has the same eager-only limitation — Python could ship the better surface here.

[bokelley]

Follow-up: lazy-factory variant landed

Three commits on this branch address your feedback:

register_lazy() + async resolve() — register_lazy(tenant_id, *, agent_url, factory, await_first_validation=False) takes a PlatformFactory = Callable[[str], Awaitable[DecisioningPlatform]]; the platform is built on the first await registry.resolve(host) hit and cached. Concurrent first-hits serialize on the per-tenant lock — one factory call per tenant. Existing resolve_by_host (sync) is unchanged and returns None until the platform is built.

Health semantics — pending = registered, factory not yet invoked. First resolve(): pending → healthy on success, pending → disabled on factory/validator failure. Factory is cleared on disabled so subsequent resolve() calls return None without retrying; recovery requires register_lazy() + recheck().

Two correctness issues found + fixed in pre-PR review:

• register_lazy(await_first_validation=True) + validator returns False: platform was being cached in _platforms and the factory was leaking into _factories. Fixed to match resolve() cold-path: discard the rejected platform, clear the factory.
• resolve() docstring implied it always returns None when the validator rejects — only true on the cold-path. Fast-path (eager or previously-resolved lazy) returns TenantResolution(health="disabled"). Docstring and inline warning updated: always check result.health, never gate solely on result is None.

Your non-blocking observations all addressed: _normalize_host load-balancer port caveat added, serve_options single-vs-multi-tenant clarification updated, lock lifecycle documented.

One nit to flag: async def resolve(host) shares its name and arity with SubdomainTenantRouter.resolve — a @runtime_checkable Protocol in this package. Mypy catches the type mismatch; duck-typing doesn't. Added a class-docstring warning. Happy to rename to resolve_platform if you'd prefer to close the pitfall before this merges.

41 tests pass.

---

Triaged by Claude Code. Session: https://claude.ai/code/session_01DRv6qahN7Jjt3Q4oxGBXkd

---

Generated by Claude Code