Skip to content

docs(api): thematic API reference generated from coverage manifest - #856

Merged
ocots merged 8 commits into
mainfrom
docs/api-reference
Aug 15, 2026
Merged

docs(api): thematic API reference generated from coverage manifest#856
ocots merged 8 commits into
mainfrom
docs/api-reference

Conversation

@ocots

@ocots ocots commented Aug 14, 2026

Copy link
Copy Markdown
Member

Summary

  • Replaces the old hand-maintained api/public.md (now in docs/attic/) with thematic API-reference pages generated at build time.
  • docs/api_reference.jl now parses the contract in docs/reports/99-api-coverage.md, builds a API_THEMES manifest, and writes one page per theme (modelling, problem, solving, options, solution, flows, geometry, types, io, qualified, internals).
  • Adds a strict coverage check: every exported symbol must appear in exactly one theme; every non-exported symbol in a theme must be in qualified. The build fails with error on drift.
  • Renames the old auto-generated Private page to Internals (per the Handbook wording) and keeps describe.jl in it.
  • Rewrites docs/src/api/ecosystem.md to list the seven control-toolbox packages with @extref links, including the missing CTLie.
  • Generated pages are created before makedocs and removed afterward by the existing _cleanup_pages mechanism, so only ecosystem.md remains tracked.

Follow-up:

  • Rebased onto the current docs/infrastructure (docs: infrastructure for v2.1 rewrite #854) tip, now that docs: infrastructure for v2.1 rewrite #854's own build issues are fixed.
  • Independently re-verified all seven of 09-api-reference.md's acceptance criteria against a real, from-scratch rebuild (julia --project=docs docs/make.jl + npx vitepress build build/1), not just the diff. All hold. This rework also fixes the 10 unresolved self-@refs for solve/methods/describe that docs: infrastructure for v2.1 rewrite #854's build was stuck showing on the old api/private.md.
  • _typos.toml and the docs/make.jl InterLinks/Draft fixes: inherited from docs: infrastructure for v2.1 rewrite #854's tip, nothing to add here.
  • A large volume of "cannot resolve @ref" warnings on the generated theme pages (79 CTBase, 59 CTModels, 17 CTFlows, 10 CTSolvers, 4 CTLie), from sibling packages' own docstrings whose "See also" cross-references use bare @ref — correct only within their own doc build, not when the docstring is reused here. Not this PR's job, and not a regression it introduces (external_modules_to_document unchanged from main), but not permanently unfixable either: the maintainer has started converting these to self-referencing @extref <Pkg> ... in CTModels (example), which resolves correctly both in the owning package's own build and when transcluded here — OptimalControl's InterLinks are already correctly wired to pick this up automatically once it lands and is released, no change needed on this side.
  • Found and fixed two related upstream/infra bugs while chasing the remaining Error: lines:
    • docs/make.jl's CTBase/CTLie/CTSolvers InterLinks entries pointed at control-toolbox.org/<Name>/..., but those three packages' sites actually deploy at <Name>.jl/... (unlike the repo names, which have no .jl suffix) — fixed on docs: infrastructure for v2.1 rewrite #854.
    • CTBase.Strategies.parameter has a real docstring in source but is missing from CTBase's own generated API reference, so no InterLinks inventory (local or published) has an entry for it — filed as CTBase#537; this is the one remaining Error: line and it's out of this repo's control.

Test plan

  • julia --project=docs docs/make.jl completes with exit 0.
  • VitePress build finishes successfully.
  • docs/src/api/ contains only ecosystem.md after the build.
  • API Reference sidebar shows the new thematic pages plus Ecosystem.
  • Reviewer check: no unintentional symbol drift (re-run names(OptimalControl) and compare with the generated pages) — the completeness check itself now enforces this on every build (zero missing, zero stale), independently re-run and confirmed.

Generated with Devin, follow-up fixes by Claude Code

Make the documentation buildable again, archive the pre-v2.1 pages,
and land the final capability-first skeleton so the remaining
content PRs do not touch the build environment.

- realign docs/Project.toml with the root [compat]
- add CTLie, DifferentiationInterface, ForwardDiff, OrdinaryDiffEqTsit5
- fix docs/make.jl: defensive extension loading, InterLinks, sitemap, Literate
- archive old docs/src/*.md and api/{public,subpackages}.md to docs/attic/
- create stub pages for the new sitemap
- track docs/src/assets/Manifest.toml and Project.toml for reproducibility

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
ocots and others added 6 commits August 15, 2026 21:37
The build was never actually run before this branch was opened (all
test-plan checkboxes were unchecked). Running it surfaced three real
defects:

- 9 stale @ref anchors in the guided-tour Literate source, still
  pointing at manual-*/example-* pages this branch moved into
  docs/attic/. Remapped to the final @id anchors this branch itself
  creates (modelling-abstract-syntax, solve-overview, flows-from-ocp,
  examples-gallery, ...).
- A MethodError in the guided tour: `saveat=` is not a keyword of any
  Hamiltonian-flow calling signature in the resolved CTFlows
  0.16.3-beta.
- Hard build errors from docs/inventories/ being an empty stub: the
  ExaModels and MadNLP entries fall through local-fallback-then-URL to
  a genuinely dead site (control-toolbox.org and exanauts/madnlp docs
  both 404 as of writing). Added honest placeholder inventories so the
  build degrades to "not found" instead of erroring.

`julia --project=docs docs/make.jl` (draft) and
`npx vitepress build build/1` both complete cleanly now.

Two residual issues are pre-existing and out of this PR's scope:
CTBase's own published objects.inv is stale/dead (external, upstream),
and 10 unresolved self-@refs for solve/methods/describe on
api/private.md trace to docs/api_reference.jl's page-generation logic,
which this PR is not permitted to touch (PR 4's mandate).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
"maintenant" (docs/reports/06-geometry.md:11) and "construit"
(docs/reports/05-flows-indirect.md:201) are correct French in quoted
spec text, not misspellings of "maintenance" / "construct". Surfaced
by CI on PR #856, but the flagged text predates it (already on main
since PR 1, the spec-set merge) — unrelated to infrastructure/shims/
api-reference scope, so fixed standalone here rather than bundled
into any of the three.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Their deployed doc sites live at control-toolbox.org/<Name>.jl/, not
control-toolbox.org/<Name>/, even though the GitHub repos themselves
have no .jl suffix (unlike CTDirect.jl/CTFlows.jl/CTModels.jl/
CTParser.jl, where repo name and site path agree). Verified directly:
the no-suffix URLs 404, the .jl-suffixed ones serve a real
objects.inv with matching content to the local build.

This was silently masked whenever a local sibling build already
existed (checked first), which is why it surfaced now: not as three
new user-visible symptoms, but as "Could not load inventory ... from
any available sources" the moment either the local build was missing
or (as in a nested git worktree) its relative path didn't resolve.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The Draft=false override made this the only page that actually ran
its @example blocks this early, and doing so surfaced 4 unrelated
runtime bugs beyond the one (saveat=) already fixed here — including
a genuine CTModelsPlots layout error, not just an API rename. The
guided tour is real tutorial content that PR 11 has not written or
debugged yet; forcing early execution contradicts this PR's own
stated scope ("a green build with mostly empty pages").

Anchor resolution is unaffected: @ref/@extref link-checking runs
independently of draft mode, so the 9-anchor remap and the ecosystem
of fixes already on this branch still apply once PR 11 re-enables
execution for this page.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Replace the hand-maintained api/public.md with theme pages generated
  from docs/reports/99-api-coverage.md at build time.
- Add missing/stale coverage checks against names(OptimalControl) so the
  reference cannot drift from the code again.
- Rename the old Private page to Internals and keep describe.jl in it.
- Rewrite api/ecosystem.md with the seven control-toolbox packages,
  including the missing CTLie link.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…rk board

Independently rebuilt (julia --project=docs docs/make.jl + npx
vitepress build build/1) rather than trusting the diff. All seven
acceptance criteria in 09-api-reference.md hold, and this rework also
fixes the 10 unresolved self-@refs for solve/methods/describe found
stuck on the old api/private.md while auditing PR 2's build.

Not this PR's job, and not a regression it introduces (same
external_modules_to_document as main): a large volume of "cannot
resolve @ref" warnings on the generated theme pages, from sibling
packages' own docstrings whose "See also" cross-references only
resolve within their own doc build. warnonly=true already tolerates
these.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ocots
ocots force-pushed the docs/api-reference branch from e372f14 to 470c59f Compare August 15, 2026 21:24
The markdown scraper (_extract_themes/_clean_token) parsing
99-api-coverage.md at build time picked up 46 symbols that don't
resolve in @docs: bare names needing module qualification
(constraint, objective, variable, parameter, ...) and six symbols
the coverage doc explicitly says are *not* documented as available
(solve_explicit, solve_descriptive, SolveMode,
display_ocp_configuration, get_full_strategy_registry,
will_solver_print) that got scraped anyway, since the parser has no
notion of negation in prose.

Replace it with a literal, hand-maintained API_THEMES array, as this
PR's own spec originally described. Qualify the symbols that need
it, drop the six not-documented ones, and move the six module-alias
symbols (CTBase, CTLie, CTFlows, CTModels, ADNLPModels, ExaModels) to
prose on qualified.md instead of @docs, since a bare module with no
docstring of its own only renders Julia's generic export listing.

One genuine upstream gap found in the process and filed as
CTDirect.jl#623: Collocation has no docstring at all.

Verified by a full rebuild: zero "undefined binding" / "no docs
found" / "duplicate docs found" warnings, down from 46 broken
symbols across 5 pages.
@ocots
ocots merged commit efdc62c into main Aug 15, 2026
6 checks passed
@ocots
ocots deleted the docs/api-reference branch August 15, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant