Skip to content

[FEAT]: Added userguide and API documentation#35

Merged
bashirpartovi merged 6 commits into
microsoft:mainfrom
bashirpartovi:dev/bashirpartovi/docs
Apr 30, 2026
Merged

[FEAT]: Added userguide and API documentation#35
bashirpartovi merged 6 commits into
microsoft:mainfrom
bashirpartovi:dev/bashirpartovi/docs

Conversation

@bashirpartovi

Copy link
Copy Markdown
Contributor

Add comprehensive documentation with MkDocs + mkdocstrings

Summary

Adds a full documentation site for RAMPART using MkDocs Material and mkdocstrings. The docs are designed for engineers consuming RAMPART to write safety tests against their own agents — not for RAMPART maintainers.

What's included

Documentation site infrastructure

  • mkdocs.yml — MkDocs configuration with Material theme, mkdocstrings Python handler, navigation structure, and Mermaid diagram support
  • docs/stylesheets/custom.css — Custom theme colors matching the RAMPART logo (dark charcoal + orange-amber)
  • pyproject.toml — Added docs dependency group (mkdocs, mkdocs-material, mkdocstrings[python]) and setuptools package-find exclusion for site/

Narrative documentation (hand-written)

Section Pages Purpose
Getting Started Installation, Quickstart, pytest Integration Setup repo, write first test, understand markers/fixtures
Concepts Overview, Attacks, Probes, PyRIT Integration Mental model, component diagram, lifecycle, evaluator polarity
Attacks Index, XPIA XPIA phases, inline vs surface-based injection, trigger options
Probes Index, Behavioral Behavioral probe usage, single/multi prompt, custom drivers
Guides Authoring Tests, Configuration, Results & Reporting, CI Integration Adapter/session/evaluator patterns, LLMConfig/Persona/AppManifest, report sinks, trial markers in CI
Glossary 25 terms Cross-referenced to API docs via mkdocstrings autorefs

API reference (auto-generated from docstrings)

12 pages using ::: mkdocstrings directives — content is pulled directly from source docstrings at build time:

  • Core Types, Core Protocols, Attacks, Probes, Evaluators, Drivers, Payloads, Surfaces, Converters, Reporting, pytest Plugin

Source docstring fixes

Updated docstrings in 5 source files to improve rendered API docs:

  • Converted RST :: literal blocks to fenced ```python blocks for proper syntax highlighting in mkdocstrings
  • Fixed PromptDriver docstring — removed inaccurate claim that all drivers are stateless and safe to reuse
  • Files changed: converter.py, payloads/__init__.py, payloads/template.py, reporting/json_file.py, pytest_plugin/plugin.py

How to build

mkdocs build    # Static site → site/
mkdocs serve    # Dev server → http://127.0.0.1:8000

Screenshot

image image

@bashirpartovi bashirpartovi requested a review from a team April 28, 2026 21:45

@spencrr spencrr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good so far - I will run this locally soon and have more commentary on the doc content - do we want to setup a github pages workflow separately? Happy to take a look!

Comment thread docs/getting-started/installation.md
Comment thread pyproject.toml
Comment thread docs/getting-started/installation.md Outdated
Comment on lines +106 to +107
!!! note
`asyncio_mode = "auto"` is recommended. It lets pytest-asyncio handle `async def test_*` functions automatically, so you don't need to decorate every test with `@pytest.mark.asyncio`. Without it, you must add that decorator to each async test individually.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we follow this in our tests?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, pyproject for one of our PoCs
image

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, I meant this repo - @pytest.mark.asyncio in RAMPART

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we don't do that in RAMPART

Comment thread docs/getting-started/quickstart.md
Comment thread docs/index.md
Comment thread docs/getting-started/installation.md Outdated
Comment thread docs/getting-started/quickstart.md
Comment thread docs/attacks/index.md
Comment thread docs/attacks/xpia.md Outdated
Comment thread docs/getting-started/quickstart.md
Comment thread docs/usage/pytest-integration.md

@rlundeen2 rlundeen2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@bashirpartovi bashirpartovi merged commit 766f71a into microsoft:main Apr 30, 2026
11 checks passed
@bashirpartovi bashirpartovi deleted the dev/bashirpartovi/docs branch April 30, 2026 18:54
@spencrr spencrr mentioned this pull request Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants