tracking: engineering & docs quality improvements — config schema / provider doc templates / architecture deep-dives / AGENTS.md / concept abstraction

[moonming]

Background

A recent cross-product (DP + CP + Dashboard) engineering and documentation audit surfaced a number of improvement opportunities across three areas: architecture clarity, documentation organization, and concept abstraction. This issue is the consolidated tracking issue, ordered by priority.

Cross-repo items (#1 / #4 / #5) also touch AISIX-Cloud — flagged inline.

---

High Priority (recommended this quarter)

1. Canonical JSON Schema as config source of truth — cross-repo

Problem: Today our config surface is fragmented:

• DP: config.example.yaml + hand-written field descriptions in docs/configuration/
• CP: Go struct + GORM AutoMigrate (28 tables), no exported schema
• Dashboard forms: hand-coded validation, prone to drift from CP/DP

Consequences: new fields get added in one place and forgotten elsewhere; dashboard validation is reimplemented; API/config docs are hand-maintained and drift over time.

Proposal: Two categories of schema (three files), plus OpenAPI generation downstream of the resource schema.

(a) Resource Schema (shared between DP and CP, single source of truth)

Covers Model / ApiKey / ProviderKey / CachePolicy / Guardrail / RateLimit / ObservabilityExporter / Routing (and any future resources).

• Location: api7/ai-gateway/schemas/ (DP is the strict consumer, so co-locating with DP keeps schema and release in sync)
• AISIX-Cloud pulls via git submodule or pinned tag, regenerates Go types
• DP uses the jsonschema crate to validate etcd watch payloads; failed payloads go to the existing dpmgr_rejected_resources table
• CP uses gojsonschema for REST API request validation; dashboard uses RJSF (or similar) to auto-render JSON Schema → React forms
• Docs auto-generate field references from schema — eliminates hand-maintained drift

(b) DP bootstrap schema — ai-gateway/schemas/bootstrap.schema.json

Covers etcd connection, proxy/admin ports, TLS, observability, cache backend, managed-mode toggle, and other process-level config.

(c) CP process config schema — AISIX-Cloud/schemas/cp-config.schema.json

Covers DB URL, JWT issuer, Stripe key, dashboard URL, etc.

Current OpenAPI state (audit before scoping (d))

A quick fact-check found a structural asymmetry that shapes the OpenAPI plan:

Aspect	AISIX-Cloud (CP)	ai-gateway (DP)
OpenAPI file	None	Hand-written, 594 lines in crates/aisix-admin/src/openapi.rs
Coverage	—	Admin API only (/admin/v1/*); proxy /v1/* deliberately excluded (defers to OpenAI's published spec)
Implementation	—	Raw JSON string literal (OpenAPI 3.1)
Tooling	None	utoipa / utoipa-axum / utoipa-scalar declared in Cargo.toml but 0 actual usage
Served at	—	GET /admin/openapi.json + GET /admin/openapi-scalar (Scalar UI)
Doc references	None	docs/configuration/admin-api.md, docs/reference/admin-api-reference.md

Two signals worth noting:

1. DP tried utoipa derive macros and walked away. The comment in openapi.rs:15-19 records the rationale: "Small enough that maintaining it by hand is less effort than wiring utoipa derive macros across every handler; the surface is stable enough that drift is easy to spot in review." The unused utoipa* deps can be cleaned up.
2. Structurally inverted. Customer-facing CP user APIs (41+ endpoints, PAT/JWT/mTLS multi-auth) are unspecified; the internal DP admin API (~10 endpoints, stable) has a spec. This is path-dependent history, not a design choice. CP work is 3-5× the DP work.

(d) OpenAPI generation (downstream of resource schema)

Once the resource JSON Schema lands ((a)), it should drive OpenAPI for both surfaces:

DP — refactor, do not rewrite:

• Keep the existing hand-written openapi.rs
• Replace inline schemas objects with $ref to the canonical resource JSON Schema files
• Single source of truth for resource shapes; eliminates DP-code / DP-OpenAPI / CP-OpenAPI triple maintenance
• Clean up unused utoipa / utoipa-axum / utoipa-scalar deps from Cargo.toml

CP — build from scratch, schema-first:

• Cannot use DP's "hand-write a small JSON" approach — CP surface is too large and varied
• Author OpenAPI 3.1 files where components/schemas $refs the canonical resource schema
• Use oapi-codegen to generate Go route handlers and DTOs from OpenAPI — compiler-enforced no-drift
• Dashboard uses openapi-typescript + openapi-fetch to generate TS types and client — replaces hand-written fetchers
• Split by audience (cleaner than tag-based separation):

File	Audience	Auth	Public?
openapi/cp-admin.yaml	Customer PAT scripts	Bearer PAT	Yes (publish SDK)
openapi/cp-dashboard.yaml	Dashboard internal	JWT (Better Auth)	No
openapi/cp-internal.yaml	DP → CP calls	mTLS	No

Stripe webhook stays out (third-party-defined contract).

Phased rollout (folded into the #1 effort estimate):

• Phase 1 (with PR #1: Scaffold Cargo workspace, UI skeleton, and CI pipeline #1 — first 2-3 weeks): land resource schema + CP customer-PAT API spec (~10 endpoints: env / model / api_key / provider_key / budget / guardrail CRUD). Validate oapi-codegen flow without migrating all handlers. Refactor DP openapi.rs to $ref.
• Phase 2 (separate, 2-4 weeks): extend to dashboard + internal API; migrate dashboard to generated client.

Versioning: Add a top-level version: 1|2 field to handle semantic-breaking changes (e.g. empty array [] meaning "allow all" in v1 vs. "deny all" in v2). Critical for forward-compatible migration.

Effort: 2-3 weeks for Phase 1 (resource schema + bootstrap schemas + CP customer-PAT OpenAPI + DP openapi.rs $ref refactor); Phase 2 separate.
Owner: platform engineer + tech writer
Touches: ai-gateway (schemas/, refactor openapi.rs, drop unused utoipa* deps), AISIX-Cloud (schemas/, 3 OpenAPI files, oapi-codegen toolchain), dashboard (generated client in Phase 2)

---

2. Provider documentation template + schema-rooted links — documentation P1

Problem: Today docs/reference/provider-compatibility.md is just a matrix table. Users diagnosing provider-specific behavior (e.g. how Anthropic upstream preserves cache_control, thinking blocks, and tool_use as byte-level passthrough) have to grep the source.

Proposal:

• Split per-provider pages for the 4 full-feature providers: OpenAI / Anthropic / Gemini / DeepSeek (Cohere / Jina rerank-only stay in the matrix)
• Fixed template per page:
  • Overview
  • Supported Operations matrix
  • For each op: Request Parameters table → field types hyperlinked to crates/aisix-core/src/models/*.rs GitHub permalinks
  • Special Message Handling (e.g. Anthropic byte-level passthrough specifics)
  • Token & Parameter Enforcement
  • Other conversions / Known caveats

Location: docs/providers/{provider}.md
Effort: 1 week
Owner: tech writer + provider crate owner review
Touches: ai-gateway docs

---

3. Architecture deep-dive documents (at least 3) — documentation P1

Problem: Platform engineers evaluating our gateway cannot find engineering-depth design explanations. docs/overview/core-concepts.md is product-flavored; there is no engineering-RFC-grade content. This is a real procurement decision blocker.

Proposal: 3 architecture docs under docs/architecture/:

1. snapshot-and-watch.md — etcd watch → ArcSwap → lock-free read path

   • Why ArcSwap instead of RwLock
   • Copy-on-write semantics
   • Memory bounds under churn
   • With Mermaid flow diagrams

2. two-phase-rate-limit.md — RPM/RPD immediate check (pre-commit) + TPM/TPD deferred counting (post-commit)

   • Design rationale (why token usage cannot be pre-estimated)
   • Failure modes and compensation
   • Semaphore-based concurrency limit integration
   • With sequence diagrams

3. protocol-translation.md — /v1/messages as a first-class endpoint

   • Anthropic upstream byte-level passthrough strategy
   • Non-Anthropic upstream (OpenAI / Gemini / DeepSeek) protocol synthesis → ChatFormat → Anthropic SSE event stream
   • Sequence diagram per case

Each doc: 3-5 Mermaid diagrams, ~2-3k words.

Effort: 1 week (one senior engineer drafts, tech writer polishes)
Owner: senior engineer + tech writer
Touches: ai-gateway docs

---

4. AGENTS.md / project-specific AI agent navigation — documentation cross-repo

Problem: Today the root CLAUDE.md is generic behavioral guidelines (think-before-code, surgical changes, etc.), not codebase navigation. AI agents (Claude Code / Cursor / Copilot etc.) entering our repos lack a project-specific entry-point document, hurting output quality.

Proposal: One file per repo.

ai-gateway/AGENTS.md covers:

• Crate topology + one-line responsibility per crate (aisix-core / aisix-etcd / aisix-gateway / aisix-proxy / aisix-admin / aisix-obs / aisix-ratelimit / aisix-cache / aisix-guardrails / aisix-server / provider crates)
• Where key traits/structs live: Bridge trait, AisixSnapshot, ProxyState
• Common task entry points: "add a provider" / "add a hook point" / "modify resource schema" / "add a route" / "wire a new metric"
• Testing entry: where unit / integration / e2e tests live, how to run them, what real backends they assume

AISIX-Cloud/AGENTS.md covers:

• Two binaries (cp-api + dp-manager) and how they share state via PG + kine
• Handler/route registration pattern (internal/cpapi/*/handlers.go)
• GORM model conventions, how to add a new table, migration discipline
• CP→DP config propagation (resource_outbox poller → kine → DP etcd watch)
• Dashboard ↔ cp-api reverse proxy, Better Auth integration
• Stripe/billing and audit log entry points

Effort: 0.5 day per repo
Owner: senior engineer + tech writer
Touches: ai-gateway, AISIX-Cloud

---

Medium Priority (consider next quarter)

5. "Virtual Key" concept repositioning — documentation cross-repo

Problem: Today our APIKey resource bundles authentication credential + allowed_models whitelist + rate_limit + team binding. The name "API Key" reads as "auth credential", but the resource is really a governance/policy entity. The industry has converged on "Virtual Key (VK)" for this concept (Stripe restricted keys, OpenRouter, Portkey, etc.).

Proposal:

• Docs: rename references to "Virtual Key" where the entity is discussed at the policy/governance level; reserve "API key string" for the literal secret value
• Dashboard UI labels: update visible labels (DB column name + Go struct stay as-is to avoid migration risk)
• Add docs/overview/glossary.md (or extend existing): "Virtual Key (VK) — a policy-bound credential combining model whitelist, rate limit, and team/budget association"

Effort: 2-3 days
Owner: PM + tech writer
Touches: ai-gateway docs, AISIX-Cloud docs, dashboard

---

6. Symmetric hook execution + tri-state fallback control — enhancement

Problem: Today our aisix-guardrails pipeline runs pre/during/post hooks in forward order. There is no guaranteed reverse-order cleanup of already-executed pre-hooks on short-circuit. Additionally, plugins/hooks have no way to signal "this short-circuit should/shouldn't trigger fallback routing."

Proposal:

• Refactor GuardrailEngine: PreHooks execute forward, PostHooks execute reverse (LIFO). On short-circuit, only already-fired PreHooks have their corresponding PostHook called (symmetric cleanup).
• Add AllowFallbacks: Option<bool> to the short-circuit response: None = default policy / Some(true) = force fallback / Some(false) = forbid fallback. The routing layer reads this to decide whether to walk the fallback chain.
• Apply the same semantics to the future plugin system.

Effort: 1 week (refactor + tests)
Owner: rust engineer
Touches: ai-gateway (aisix-guardrails, aisix-proxy)

---

7. Reserved header namespace documentation — documentation

Problem: We already emit x-aisix-cache: hit|miss, but no canonical doc spells out: (a) which x-aisix-* response headers the gateway emits; (b) which x-aisix-* request headers clients can set to control gateway behavior; (c) the reserved-prefix contract (what gets stripped vs. forwarded upstream).

Proposal: Add a "Reserved Header Namespace" section to docs/reference/headers-and-error-codes.md:

• Declare x-aisix-* as the gateway's reserved prefix
• List all response headers the gateway emits (request-id, cache status, which fallback target was used, etc.)
• List all request headers clients can actively set (cache TTL override, disable cache, force/disable fallback, etc.)
• Clarify which headers are stripped before forwarding upstream (Authorization and other sensitive headers)

Effort: 0.5-1 day
Owner: tech writer + engineer review
Touches: ai-gateway docs

---

8. Error messages with embedded operational guidance (doc-as-error) — enhancement documentation

Problem: Today crates/aisix-core/src/errors.rs and various error responses describe "what went wrong" but not "what to do next." Industry best practice: error messages include the actionable next step ("set X in config" / "in dashboard go to Y > Z").

Proposal:

• Audit every user-facing error variant
• For each, add a "next step" sentence covering: the config key to change, the dashboard path to navigate, or the doc URL to read
• Put the text directly in the error Display impl — no dependency on external docs

Example transformation:

• Before: "request timed out"
• After: "request timed out after 30s. Increase proxy.request_timeout_seconds in bootstrap config, or override per-model via Dashboard > Models > {model} > Network Config."

Effort: 1-2 days
Owner: backend + tech writer
Touches: ai-gateway core, ai-gateway docs (cross-reference)

---

9. CLI agent integration cookbook — documentation

Problem: Users adopting AI coding assistants (Claude Code, Cursor, Codex CLI, Zed, OpenCode, LibreChat, etc.) need step-by-step recipes for pointing those clients at our gateway. Today we only have OpenAI/Anthropic SDK quickstarts — no client-specific recipes. This is a high-SEO, high-conversion content category.

Proposal: Add docs/integrations/cli-agents/ (or docs/cookbook/cli-agents/) with at least 3-5 recipes:

• Claude Code → ai-gateway
• Cursor → ai-gateway
• Codex CLI / Aider → ai-gateway
• Zed → ai-gateway
• LibreChat / Open-WebUI → ai-gateway

Each recipe: ~500-800 words, with config snippet, verification step, common pitfalls.

Effort: 1 week (3-5 recipes)
Owner: tech writer + DevRel
Touches: ai-gateway docs

---

Strategic / Forward-Looking

10. MCP support — P1 enhancement

Already on the P1 roadmap (#58). Scope reminder:

• MCP client mode (gateway connects to external MCP servers)
• MCP server mode (gateway exposes tools to MCP-aware clients like Claude Desktop)
• Agent Mode with auto-execute tool whitelist (selected tools execute without per-call approval)
• Forward-looking: Code Mode — a sandboxed Python/Starlark execution layer that lets the LLM author multi-tool orchestrations (vs. multi-turn tool_call loops). Empirical reports from comparable products: -50% tokens, -40% latency vs. multi-turn. Recommend storing this design in the PRD as a candidate for future differentiation; do not block initial MCP delivery on it.

Owner: PM + senior engineer (design phase)

---

11. Multi-DP cluster patterns (strategic observation only — no action yet)

Current architecture: DP nodes are stateless, kine-on-PG is the single source of truth — sufficient for current scale.

When multi-region DP federation or autonomous DP clusters become a real requirement, evaluate split-transport cluster patterns: gossip for membership/liveness (low-churn, short messages) + gRPC for application state sync (higher-throughput, structured payloads). No action until the requirement materializes.

---

Anti-patterns to actively avoid (architecture-alignment notes)

For team alignment, we explicitly do NOT plan to adopt:

1. Monolithic Provider interface — our Bridge trait (4-6 methods) is cleaner than putting OCR/Container/CachedContent/Speech/Batch all on a single interface. Document this as a deliberate design choice in the architecture doc series (feat(core): Model, ApiKey, RateLimit entities + JSON Schema validators #3).
2. DP-bundled persistence SDK — DP stays stateless; ConfigStore / LogStore / VectorStore must live on the CP side.
3. DP-bundled Web UI — UI is a Cloud-only concept; DP exposes admin API only. This is a hard requirement of the multi-tenant architecture.
4. More than 4 budget tiers — Org → Env → APIKey → ProviderKey is sufficient.

---

Action item summary

#	Item	Effort	Priority	Owner	Touches
1	Canonical JSON Schema (resource + 2× bootstrap) + OpenAPI generation (DP refactor, CP from scratch)	2-3 wk (Phase 1)	High	platform eng	DP+CP+Dashboard
2	Provider docs template + schema links (4 pages)	1 wk	High	tech writer	DP docs
3	Architecture deep-dives (3 docs)	1 wk	High	sr eng + writer	DP docs
4	AGENTS.md per repo	0.5d × 2	High	sr eng + writer	DP + CP
5	Virtual Key concept repositioning	2-3 d	Medium	PM + writer	docs + UI
6	Symmetric hook + tri-state fallback	1 wk	Medium	rust eng	DP
7	Reserved header namespace doc	0.5-1 d	Medium	writer + eng	DP docs
8	Doc-as-error audit	1-2 d	Medium	backend + writer	DP
9	CLI agent integration cookbook (3-5 recipes)	1 wk	Medium	writer + DevRel	DP docs
10	MCP support (already on P1 roadmap)	—	Strategic	PM + sr eng	DP
11	Multi-DP cluster patterns	—	Watch only	—	—

---

JSON Schema placement (clarification on #1)

The resource schema in #1 is a single file (not one per side). If DP and CP each maintain their own copy, drift over time is inevitable as fields are added.

• Resource schema = 1 file, shared between DP and CP, lives in the ai-gateway repo
• DP bootstrap schema = 1 file, lives in the ai-gateway repo
• CP process config schema = 1 file, lives in the AISIX-Cloud repo

Three independent files, no overlap.

[coderabbitai]

🔗 Related PRs

api7/api7ee-3-control-plane#2369 - docs: add hierarchical AGENTS.md knowledge base for AI coding agents [merged]
api7/docs#1436 - docs: add AI Gateway documentation [merged]
api7/docs#1452 - docs: restructure API7 API Gateway documentation [merged]
api7/aisix-ee#3 - feat: CP mode — heartbeat agent, mTLS client, instance_id persistence [merged]
api7/api7ee-3-control-plane#2627 - fix: adapt aisix provider/model two-tier architecture and add Provider CRUD API [merged]
#280 - feat: add RateLimitPolicy entity and policy-based rate limiting [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!