docs(api): API reference docs umbrella (AGE-3734)#4314
Merged
Conversation
Introduce a shared concept page documenting the artifact / variant / revision pattern used by applications, workflows, evaluators, and testsets. Covers commit semantics, retrieval references, revision log, and archive/unarchive. Factors out a pattern that would otherwise have to be repeated in every per-resource reference page. The applications, workflows, evaluators, and testsets concept pages (landing in parallel PRs) link to this page instead of redefining the model.
Adds the Evaluators concept page under the REST API guide and annotates every evaluators endpoint and request/response model with docstrings plus Field(description=...). - New: docs/docs/reference/api-guide/07-evaluators.mdx - Adds handler docstrings to EvaluatorsRouter and SimpleEvaluatorsRouter (catalog, artifact, variant, revision, deploy, resolve, simple CRUD, templates) - Adds Field descriptions to every request/response schema in api/oss/src/apis/fastapi/evaluators/models.py No behavior changes; no openapi_extra; no marketing words. Cross-links to Versioning and Query Pattern guides.
- Add `docs/docs/reference/api-guide/05-applications.mdx` covering what an application is, versioning, invocation, deployment, simple vs structured endpoints, catalog, lifecycle, and an end-to-end worked example. - Add docstrings to all 35 Applications FastAPI handlers (29 structured + 6 simple), explaining what each endpoint does and when to pick it over a similar one. - Add `Field(description=...)` to every field on the applications request and response models, including the catalog and simple-application models. - Every endpoint was exercised against `eu.cloud.agenta.ai` before its docstring was written. - Hooks skipped in sandbox; will be run locally before merge. No secrets in this PR (docs + docstrings only).
- Adds 08-testsets.mdx — concept page covering immutable testcases, the versioned-bag model, dedup paths (testcase_dedup_id / __dedup_id__ legacy), import/export, simple testsets, and pinning in evaluations. - Adds Field descriptions on testsets/testcases models and docstrings on every router handler.
- Adds 06-workflows.mdx — concept page covering workflows as the versioned-logic primitive, when to pick workflows vs applications vs evaluators, the artifact/variant/revision model, and the shared shape with applications and evaluators. - Adds Field descriptions on workflows models and docstrings on every router handler.
- Adds 10-tracing.mdx — concept page covering spans/traces, the invocation vs annotation trace types, span types, references vs links, the four ingestion paths with a decision tree, the async 202 contract, the ag.* namespace (both wire formats), the ag.unsupported fallback, and the query focus mode. - Adds Field descriptions across tracing/, traces/, otlp/ models and docstrings on every handler beyond the PR #4174 baseline.
- Note that revisions carry the payload, not just commit metadata. - Add environments as a deployment pointer alongside artifact/variant/revision. - Reference the existing /concepts/concepts overview page. - Replace temperature-only example with a realistic application revision payload (prompt, messages, llm_config). - Explain what a reference is (id, slug, slug+version) before describing retrieve precedence. - Spell out "most specific wins" as revision > variant > environment. - Rename "Revision log" to "Listing revision history" so the heading matches the verb pattern used by the other sections. - Clarify the fork verb ties to POST /variants/fork.
Aggregates the 8 sub-PRs that add concept pages and endpoint docstrings for the REST API reference. Sub-PRs base off this branch so forward links (versioning <-> applications, workflows, evaluators, testsets, tracing, folders, evaluations) resolve during docs build. When all sub-PRs are merged into this branch, this PR merges into main.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR modernizes Pydantic API model declarations across ten FastAPI subsystems by replacing bare field defaults with explicit ChangesAPI Schema and Documentation Modernization
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Comment |
The ag_config wrapper was removed by migration 91d3b4a8c27f_fix_ag_config
(2025-04-24); revision payloads now live under data.parameters directly.
Verified against eu.cloud.agenta.ai: posting with the old ag_config
envelope returns HTTP 200 but stores data: {} because extra fields are
silently dropped. The corrected shape (data.parameters.prompt.*) stores
the payload correctly.
docs(docs): add versioning concept page
…plications-endpoints Resolve conflict in api/oss/src/apis/fastapi/applications/models.py: response envelopes now extend the new Support base class (from oss.src.utils.exceptions) while keeping the documentation docstrings and Field(description=...) introduced on this branch. No conflicts elsewhere. Router and concept page pick up cleanly from main.
…lders-endpoints Bring in main + the landed versioning concept page so the folders PR can ship on top of the umbrella branch. Conflict resolution: - api/oss/src/apis/fastapi/folders/models.py: the umbrella switched the three response models (FolderResponse, FoldersResponse, FolderIdResponse) to inherit from `Support` so they carry support_id / support_ts. Combined with the Field(description=...) annotations from this branch.
Run pnpm clean-api-docs && pnpm gen-api-docs to refresh the generated *.api.mdx pages against the current openapi.json. Picks up the support_id / support_ts fields on response envelopes and other minor regen drift.
`POST /applications/variants/fork` now raises `400 Bad Request` when the fork target is invalid (introduced in 77cb317 "workflow-related fork issues"). Add a short note to the handler docstring so the generated OpenAPI reflects the failure mode. No behavioral change.
Drop the docs/docs/reference/api/*.api.mdx regeneration from this PR. Those files are auto-generated from openapi.json and produce a lot of noise in domain PRs. They will be regenerated at a later stage in the workflow, once openapi.json itself is updated from the branch code (not from production, which is what update-api-docs does by default). Leaves the actual docstring + Field(description=...) changes in api/oss/src/apis/fastapi/folders/ intact.
docs(api): folders endpoint docstrings + concept page
…' into docs/age-3734-applications-endpoints
Rewrites the intro to lead with the customer-facing model (chat / completion endpoint), introduces the built-in application types up front, and links custom workflows as another kind of application. Splits the `data` payload section so `uri` + `schemas` are explained as drivers of execution before the catalog is referenced. Removes remaining em dashes.
…dpoints docs(api): applications endpoint docstrings + concept page
Resolve conflicts in testsets/models.py and testcases/models.py: - Keep Field(description=...) annotations from HEAD - Adopt Support base class from umbrella for response envelopes - Add Support import to both modules
Resolves Support-mixin conflicts in api/oss/src/apis/fastapi/evaluators/models.py by keeping the docstrings and Field(description=...) descriptions while adopting the umbrella's Support base class on 16 response envelopes. Reverts auto-generated docs/docs/reference/api/ changes per write-docs §8. The auto-regen is handled separately, downstream of the openapi.json update.
The evaluations concept page (PR-3 in the AGE-3734 plan) has not landed yet. Three pages (evaluators, testsets, tracing) link forward to /reference/api-guide/evaluations, which fails the Docusaurus broken-link check. For now, drop the link syntax on those three references. Two become plain bold text; the tracing cross-ref folds the pointer into a plain sentence. Restore the links in the evaluations PR when it lands.
dosubot
Bot
added
size:XXL
Contributor
Railway Preview Environment
Updated at 2026-05-14T10:13:54.899Z |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
api/oss/src/apis/fastapi/traces/models.py (1)
92-95: ⚡ Quick winUse
default_factory=listfortracesfield.Pydantic v2 recommends
default_factory=listoverdefault=[]for mutable defaults to avoid unintended side effects and schema issues. Most API models in this codebase already follow this pattern.Suggested change
- traces: List[SimpleTrace] = Field( - default=[], + traces: List[SimpleTrace] = Field( + default_factory=list, description="The matching traces in the high-level `SimpleTrace` shape.", )
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: aa2d086e-d76b-4208-8ea4-0172806446f7
📒 Files selected for processing (25)
api/oss/src/apis/fastapi/applications/models.pyapi/oss/src/apis/fastapi/applications/router.pyapi/oss/src/apis/fastapi/evaluators/models.pyapi/oss/src/apis/fastapi/evaluators/router.pyapi/oss/src/apis/fastapi/folders/models.pyapi/oss/src/apis/fastapi/folders/router.pyapi/oss/src/apis/fastapi/otlp/models.pyapi/oss/src/apis/fastapi/otlp/router.pyapi/oss/src/apis/fastapi/testcases/models.pyapi/oss/src/apis/fastapi/testsets/models.pyapi/oss/src/apis/fastapi/testsets/router.pyapi/oss/src/apis/fastapi/traces/models.pyapi/oss/src/apis/fastapi/traces/router.pyapi/oss/src/apis/fastapi/tracing/models.pyapi/oss/src/apis/fastapi/tracing/router.pyapi/oss/src/apis/fastapi/workflows/models.pyapi/oss/src/apis/fastapi/workflows/router.pyapi/oss/src/core/folders/types.pydocs/docs/reference/api-guide/04-versioning.mdxdocs/docs/reference/api-guide/05-applications.mdxdocs/docs/reference/api-guide/06-workflows.mdxdocs/docs/reference/api-guide/07-evaluators.mdxdocs/docs/reference/api-guide/08-testsets.mdxdocs/docs/reference/api-guide/10-tracing.mdxdocs/docs/reference/api-guide/11-folders.mdx
jp-agenta
reviewed
May 14, 2026
jp-agenta
reviewed
May 14, 2026
jp-agenta
reviewed
May 14, 2026
jp-agenta
reviewed
May 14, 2026
jp-agenta
reviewed
May 14, 2026
jp-agenta
reviewed
May 14, 2026
jp-agenta
reviewed
May 14, 2026
ashrafchowdury
approved these changes
May 14, 2026
Contributor
|
Nice explanation, thank you for that @mmabrouk It would be better if we had a description within each of the api references page, so that we don't have to look for an explanation - good for both human an agents. But this is good as well |
Member
Author
|
@ashrafchowdury the docstring will be autogenerated after merge into an update api reference doc |
Documentation: - 04-versioning.mdx: include environments in the shared versioning model per JP's note on the intro. - 05-applications.mdx: replace the JS-style placeholder comment in the final invoke example with valid JSON plus an inline note (CodeRabbit). - 10-tracing.mdx: rewrite the trace-types table so annotations are evaluator-produced as well as human-produced; add an :::info block showing the fields that identify each type (JP + mmabrouk on L93). - 10-tracing.mdx: mark `/tracing/spans/ingest`, `/tracing/traces/query`, `/tracing/spans/query`, and `/traces/ingest` as deprecated; point callers to `/otlp/v1/traces`, `/simple/traces/`, `/spans/query`, and `/traces/query`. Drop the deprecated nested-trace curl example and point to the SDK / OTel collector setup instead (JP on L108). API metadata (no behaviour change): - applications/models.py: reword the edit Field descriptions on ApplicationVariantEditRequest and ApplicationRevisionEditRequest. Edit is a full PUT; documenting it as "fields to update" is misleading (JP on models.py:197 and 287). - evaluators/models.py, workflows/models.py, testsets/models.py: drop the "Defaults to false" wording on the `include_archived` Field descriptions. The schema default is null, not false (CodeRabbit on evaluators/models.py:85 and the same pattern across the three domains). - tracing/router.py: apply `deprecated=True` to the remaining TracingRouter routes (`/spans/query`, `/analytics/query`, `/traces/*`, `/sessions/query`, `/users/query`). The pattern was already in place for `/spans/ingest`; this just propagates it. Out of scope here: - The count-semantics bug at tracing/router.py:296 (CodeRabbit) is a business-logic change. Filed as AGE-3758 and intentionally left untouched in this docs PR.
Member
Author
Review pass: changes summaryAddressed feedback from @jp-agenta, @ashrafchowdury, and Documentation
API metadata (no behaviour change)
Held for separate follow-up
|
Drop the "deprecated" framing on `__dedup_id__`. Per @jp-agenta on #4314 (08-testsets.mdx:35), the `legacy_` prefix in the input normalisation code is unfortunate naming; the field is not deprecated. - `data.testcase_dedup_id` is the JSON-payload key (commits, SDK). - `__dedup_id__` is the metadata-column key, alongside `__id__`, `__flags__`, `__tags__`, and `__meta__`, used by CSV uploads and downloads. Both round-trip through the server's normalisation, so callers can commit through one path and download through the other and the dedup ID stays consistent. Also add `__dedup_id__` to the metadata-column list in the Import and export section so the family of `__X__` columns is documented in one place.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Umbrella PR aggregating the AGE-3734 API reference documentation effort. The 8 sub-PRs below base off this branch instead of
mainso that forward links between concept pages (versioning ↔ applications, workflows, evaluators, testsets, tracing, folders, evaluations) resolve during the Docusaurus build.Each sub-PR contains a concept page plus the FastAPI handler docstrings and
Field(description=...)updates for its domain. Style guide:.claude/skills/write-docs/SKILL.md. Project plan:tmp-docs-analysis/plan.md.Sub-PRs
Merge flow
Test plan
npm run buildindocs/passes with no broken-link errorsruff formatandruff checkclean inapi/