DOC improve API reference accuracy, structure, and readability

[biefan]

Summary

This updates build_scripts/gen_api_md.py and build_scripts/pydoc2json.py to make generated API docs more complete, accurate, and readable.

What changed

gen_api_md.py — major refactor:

• Generate pages only for top-level public modules (reduced from 324 pages to ~28), eliminating duplicate content where submodules re-export the same
  classes as their parent.
• Recursively expand pure-aggregate modules (e.g. pyrit.executor) into their children that have actual API surface.
• Resolve aliases to full class/function definitions using a definition index, so re-exported classes render with full documentation instead of bare
  "Alias of X" stubs.
• Filter out external imports (stdlib, third-party) from rendered pages — only show members actually defined in each module's tree.
• Exclude pyrit.backend from API docs (__all__ = [], exports nothing).
• Simplify sidebar headings to show only class/function names (no type signatures), with full signatures in code blocks below.
• Wrap doctest-style >>> examples in code fences for proper rendering.
• Base member counts in logs and index preview on rendered kinds only (class/function).

pydoc2json.py — alias resolution fix:

• Add _resolve_alias_from_source fallback using griffe.visit() to parse individual .py files when griffe cannot resolve aliases due to missing
  __init__.py in namespace packages (e.g. pyrit.datasets.seed_datasets).

Banner modules marked private:

• pyrit/cli/banner.py → pyrit/cli/_banner.py
• pyrit/cli/banner_assets.py → pyrit/cli/_banner_assets.py
• Updated all imports and mock patch paths.

doc/myst.yml TOC updated to match generated pages.

Why

• Fixes missing module pages that caused broken links from doc/api/index.md.
• Eliminates duplicate API content (same class appearing on parent and submodule pages).
• Resolves previously-stubbed classes (SeedDatasetProvider, TextJailBreak, scorers, targets) to full documentation.
• Cleans up the sidebar for better navigation.
• Marks internal CLI modules as private per Python convention.

Validation

• uv run --with ruff ruff check build_scripts/gen_api_md.py build_scripts/pydoc2json.py
• pytest tests/unit/cli/test_banner.py tests/unit/cli/test_pyrit_shell.py — 70 tests pass
• Local doc site build and visual inspection

Fixes #653

[biefan]

@microsoft-github-policy-service agree

[romanlutz]

Looks good! Just want to try it out locally before merging

[Copilot]

Pull request overview

Updates the documentation build script that generates MyST API reference markdown from griffe JSON so that re-exported names (aliases) are represented and modules with no detected public members are handled more explicitly.

Changes:

• Add alias rendering and an “Aliases” section to per-module API pages.
• Generate per-module pages for all collected modules and show an explicit “no public API members” message when empty.
• Improve index previews to include functions and aliases (not just classes), with a fallback message for empty modules.

---

You can also share your feedback on Copilot code review. Take the survey.

[romanlutz]

While trying this out I had a number of additional thoughts and kept updating until I was happy. I hope you don't mind. I figured since this addresses an issue I initially filed it's probably fine.

left sidebar:

right sidebar:

code samples in api ref:

[biefan]

Ini telah ditemui dan diperbaiki oleh CODEX sendiri. Saya akan terus menghantar permintaan tarik kepada CODEX.