Skip to content

feat(cli): env-selectable embedder — offline sentence-transformers path (closes #2) - #3

Merged
witt3rd merged 2 commits into
mainfrom
cli-offline-embedder
Jun 2, 2026
Merged

feat(cli): env-selectable embedder — offline sentence-transformers path (closes #2)#3
witt3rd merged 2 commits into
mainfrom
cli-offline-embedder

Conversation

@witt3rd

@witt3rd witt3rd commented Jun 1, 2026

Copy link
Copy Markdown
Owner

Closes #2.

What

make_memory only wired the LiteLLM default embedder, so every write-path CLI invocation (retain/index/sweep) required an OpenAI key + network. The library already ships an offline embedder (prospecta.embed.sentence_transformers, 384-dim, no key) but nothing selected it from the CLI.

Changes

  • cli/_common.py — new _resolve_embedder() reads PROSPECTA_EMBEDDER (case-insensitive): unset/default/litellm → existing defaults.make_default_embedder() (unchanged); sentence-transformers/stembed.sentence_transformers(model) with PROSPECTA_EMBED_MODEL (default all-MiniLM-L6-v2). Unknown values raise SystemExit(1) with the valid set — no silent fallback, since a wrong-embedder mismatch corrupts a bank's vector space.
  • make_memory now resolves embed and llm independently. Previously llm resolved first inside the same try, so a missing litellm aborted before embed. Decoupled — an offline embedder works even when litellm is absent. Paired with retain --index-text "..." (P4, skips LLM index_text generation), this is a fully offline, keyless retain.
  • tests/test_cli.py — 5 tests: default→litellm, explicit litellm, st routes to the offline factory with default + custom model, unknown errors loud.
  • README — "Offline / keyless retain" quickstart section + the dim-must-match caveat.

No breaking change — default behavior is identical when PROSPECTA_EMBEDDER is unset.

Verification

Verified end-to-end against a live substrate, no API key:

prospecta create-bank --id tui-st --embedding-dim 384
PROSPECTA_EMBEDDER=sentence-transformers PROSPECTA_BANK=tui-st \
  prospecta retain "..." --source issue-2-verification \
  --index-text "How does the offline embedder path work?" \
  --index-text "What did Forge build for issue 2?"
# → document c525059d-... landed

Substrate confirms: source issue-2-verification, 2 memory_items, dim=384 (sentence-transformers, not the dim-32 hash), llm_generated=false (caller-supplied index_text, no LLM call). The sentence-transformers model encoded the vectors locally.

24 tests pass (test_cli + test_embed_helpers).

Why this is filed from the prospecta-tui side

The TUI manual-ops surface (#1) shells out to prospecta retain for its retain form. Without an offline embedder, that path can't be live-verified without provisioning an OpenAI key + dim-1536 bank. This unblocks end-to-end verification of the TUI manual ops — and, more broadly, gives prospecta a keyless local-dev/CI retain path.

⚒️ Forge

Closes #2.

make_memory only wired the LiteLLM default embedder, so every write-path
CLI invocation (retain/index/sweep) required an OpenAI key + network. The
library already ships an offline embedder (prospecta.embed.sentence_-
transformers, 384-dim, no key) but nothing selected it from the CLI.

cli/_common.py:
- New _resolve_embedder() reads PROSPECTA_EMBEDDER (case-insensitive):
  unset/default/litellm → defaults.make_default_embedder() (unchanged
  behavior); sentence-transformers/st → embed.sentence_transformers(model)
  with PROSPECTA_EMBED_MODEL (default all-MiniLM-L6-v2). Unknown values
  raise SystemExit(1) with the valid set — no silent fallback, since a
  wrong-embedder mismatch corrupts a bank's vector space.
- make_memory now resolves embed and llm INDEPENDENTLY. Previously llm
  resolved first inside the same try, so a missing litellm aborted before
  embed. Decoupled: an offline embedder works even with litellm absent.
  Paired with `retain --index-text "..."` (P4, skips LLM index_text gen),
  this is a fully offline, keyless retain.

No breaking change — default behavior is identical when PROSPECTA_EMBEDDER
is unset.

tests/test_cli.py: 5 tests — default→litellm, explicit litellm, st routes
to the offline factory with default + custom model, unknown errors loud.

README: "Offline / keyless retain" quickstart section + the dim-must-match
caveat (a 384-dim sentence-transformers bank is not interchangeable with a
1536-dim OpenAI bank).

Verified end-to-end against a live substrate: created a dim-384 bank,
PROSPECTA_EMBEDDER=sentence-transformers retain with --index-text landed a
real document (2 memory_items, dim=384, llm_generated=false) with no API
key — the sentence-transformers model encoded the vectors locally.

24 tests pass (test_cli + test_embed_helpers).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an environment-selectable embedder to the CLI so write-path commands can run fully offline (sentence-transformers) without requiring LiteLLM/OpenAI, aligning CLI behavior with the already-shipped offline embed helper and unblocking keyless dev/CI flows.

Changes:

  • Introduces _resolve_embedder() in prospecta.cli._common to select an embedder via PROSPECTA_EMBEDDER (+ PROSPECTA_EMBED_MODEL for sentence-transformers).
  • Updates make_memory to resolve embed and llm independently so offline embedding can work even when LiteLLM isn’t installed.
  • Adds CLI tests for embedder selection and documents an “Offline / keyless retain” workflow in the README.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
prospecta/cli/_common.py Adds env-based embedder resolution and decouples embedder vs LLM construction in make_memory.
tests/test_cli.py Adds coverage for default/litellm/sentence-transformers embedder resolution and unknown-value failure.
README.md Documents the offline/keyless retain path and dimensionality caveat.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread prospecta/cli/_common.py
Comment thread prospecta/cli/_common.py
Addresses both Copilot review comments on #3.

1. make_memory swallowed ImportError from _resolve_embedder, hiding the
   actionable install hint when a user selects PROSPECTA_EMBEDDER=
   sentence-transformers but hasn't installed the extra — it devolved into
   a confusing downstream RuntimeError about Memory(embed=...). Now the
   offline branch catches its own ImportError and re-raises SystemExit(1)
   with the factory's install hint. The default/litellm path keeps its
   soft-fail (stats/config must work without litellm), so the behavior is
   asymmetric by design and documented as such.

2. The unknown-value error message omitted the `st` alias the resolver
   actually accepts, disagreeing with the README. Now reads:
   "valid: default | litellm | sentence-transformers (alias: st)".

tests: +4 — unknown lists st alias; explicit-st-missing fails loud with the
install hint; make_memory propagates that SystemExit (doesn't swallow it).

Re-verified live: happy-path offline retain still lands (doc e02f2385,
dim-384, no key); unknown value prints the corrected message.
@witt3rd
witt3rd merged commit 43483cd into main Jun 2, 2026
@witt3rd
witt3rd deleted the cli-offline-embedder branch June 2, 2026 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CLI: env-selectable embedder (offline sentence-transformers path)

3 participants