Add retrieval docs, providers section, and example by chris-colinsky · Pull Request #220 · LunarCommand/openarmature-python

chris-colinsky · 2026-07-18T17:28:36Z

Documents the retrieval-provider capability for v0.16.0 and adds a
runnable example, mirroring the LLM documentation layout. Gating for the
release: the capability shipped in code but had no user-facing docs.

Docs

concepts/retrieval.md (new): embedding, reranking, the
input_type query/document knob, batch chunking, and the
nullable-usage contract. Concept-focused, points to the providers
section for the catalog and hosting.
retrieval-providers/ (new section, parallel to model-providers/):
- index.md: the bundled-providers table, the EmbeddingProvider /
  RerankProvider contract, behaviour guarantees, error categories, a
  minimal example.
- tei.md: self-hosting Text Embeddings Inference for embeddings and
  reranking, from a first docker run through a systemd production
  deployment, and wiring the TEI providers to it.
- authoring.md: a provider skeleton, contract checklist, and the
  chunking / input_type / observability concerns.
reference/retrieval.md (new): the API reference.
Nav entries for both new sections and a retrieval entry in the README
capabilities list.

Example

examples/retrieval-rag/: a lunar-corpus retrieval-augmented answering
pipeline. It batch-embeds a corpus into an index, then per query embeds
the question, retrieves by cosine similarity, reranks the shortlist with
a cross-encoder, and grounds an answer in the reranked passages. An
attached observer prints the embedding and rerank events, so retrieval is
a visible boundary in telemetry. Live-tested against OpenAI + Cohere;
adversarially reviewed for the provider lifecycle and the index
dependency.

Source

Exports the four retrieval event types (EmbeddingEvent,
EmbeddingFailedEvent, RerankEvent, RerankFailedEvent) top-level from
openarmature.graph, matching the LLM events, so an observer over
retrieval calls imports them the same way as one over LLM calls. Additive.

Testing

Full suite 1688 passed; mkdocs build --strict clean. The example's
build_graph() is exercised by the examples smoke test. The docs code
snippets were verified by hand against the live API (imports, constructor
fields); wiring the retrieval-providers/ snippets into the docs drift
check is a tracked follow-up.

Merge note

This branch was cut before the v0.16.0 version bump on the release-prep
branch, so it carries version 0.15.0 in the regenerated AGENTS.md. After
the release-prep PR merges, this branch rebases on fresh main and
regenerates AGENTS.md so the version lands at 0.16.0; the AGENTS.md drift
test guards against a mismatch.

Copilot

Pull request overview

Adds first-class user-facing documentation for the retrieval-provider capability (concepts, provider catalog/hosting, and API reference), plus a runnable retrieval RAG example, and aligns observability imports by re-exporting retrieval events from openarmature.graph.

Changes:

Add new Retrieval docs: concept page, retrieval providers section (including TEI self-hosting + authoring guide), and openarmature.retrieval reference page.
Add a new runnable examples/retrieval-rag/ demo and include it in the examples smoke test + examples index docs.
Re-export retrieval provider events (Embedding*, Rerank*) from openarmature.graph to match the LLM event import pattern.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test_examples_smoke.py	Adds `retrieval-rag` to the examples smoke-test list.
src/openarmature/graph/init.py	Re-exports retrieval events at the top-level `openarmature.graph` API.
src/openarmature/AGENTS.md	Lists the new `retrieval-rag` example in shipped examples.
README.md	Adds “retrieval” to the Concepts list in “Next steps”.
mkdocs.yml	Adds nav + llms.txt section entries for retrieval docs and providers.
examples/retrieval-rag/main.py	New runnable retrieval-augmented answering example using embeddings + rerank + LLM.
examples/README.md	Adds a “Retrieval” section describing the new example.
docs/retrieval-providers/tei.md	New guide for self-hosting TEI and wiring TEI providers.
docs/retrieval-providers/index.md	New retrieval providers overview: shipped providers, protocol contract, errors, minimal example.
docs/retrieval-providers/authoring.md	New provider authoring guide with skeleton + checklist.
docs/reference/retrieval.md	New mkdocstrings reference entry for `openarmature.retrieval`.
docs/concepts/retrieval.md	New retrieval concept page (embedding, rerank, `input_type`, chunking, usage, observability).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Document the retrieval-provider capability for v0.16.0, mirroring the LLM documentation layout. Concept page (concepts/retrieval.md): embedding and reranking, the input_type query/document knob, batch chunking, and the nullable-usage contract. Stays concept-focused and points to the providers section. Retrieval Providers section (retrieval-providers/), parallel to model-providers/: index.md (the bundled-providers catalog, the EmbeddingProvider / RerankProvider contract, error categories, a minimal example), tei.md (self-hosting Text Embeddings Inference for embeddings and reranking, from a first docker run to a systemd production deployment, with the providers wired to it), and authoring.md (a provider skeleton, contract checklist, and the chunking / input_type / observability concerns). API reference (reference/retrieval.md) and nav entries for both new sections; a retrieval entry in the README capabilities list. Example (examples/retrieval-rag): a lunar-corpus retrieval-augmented answering pipeline that batch-embeds a corpus, embeds the query, retrieves by cosine similarity, reranks the shortlist, and grounds an answer in the reranked passages, with an observer printing the embedding and rerank events. Exports the four retrieval event types (EmbeddingEvent, EmbeddingFailedEvent, RerankEvent, RerankFailedEvent) top-level from openarmature.graph, matching the LLM events, so observers over retrieval calls import them the same way. Regenerates the bundled AGENTS.md example index.

Address the review on PR #220. Wrap each retrieval-rag CORPUS passage in parentheses so the multi-line string is an explicit grouped expression, not adjacent string literals implicitly concatenated inside the list (a CodeQL missing-comma footgun in a copy-me example). Add the 404 -> ProviderInvalidModel branch and import to the authoring-guide skeleton's _classify(), which contradicted its own contract checklist.

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Align the retrieval-rag SearchState list defaults with the codebase's Field(default_factory=...) idiom (PR #220 review). Use the typed default_factory=list[int] rather than a bare default_factory=list, since the bare form infers list[Unknown] and trips strict pyright's reportUnknownVariableType on a list[int] field.

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

The Next Steps 'Examples' bullet hard-coded 'ten runnable demos', which was already stale and this PR adds another. Remove the fixed number so it stops drifting as examples change (PR #220 review).

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

The authoring-guide embed() skeleton parsed the 200 response body without guarding it, so a malformed body would leak a raw KeyError/ValueError instead of the ProviderInvalidResponse its own contract checklist prescribes. Wrap the parse (PR #220 review).

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 18, 2026 17:28

Copilot started reviewing on behalf of chris-colinsky July 18, 2026 17:29 View session

github-advanced-security AI found potential problems Jul 18, 2026

View reviewed changes

github-code-quality Bot found potential problems Jul 18, 2026

View reviewed changes