loom

The context-engineering layer for an agent — chunking, embeddings, vector retrieval, history compaction, and token-budgeted context assembly, in ~2k readable lines of Python.

loom answers the question every agent quietly faces on each turn: given a big pile of documents, a long conversation, and a fixed token budget, what exact context do you put in front of the model? It is not a framework and not a vector-database wrapper — it is a tight, legible implementation of the moving parts that decide what goes in the window: cut documents into chunks, embed them, retrieve the relevant ones (with optional diversity re-ranking), compact old conversation into a rolling summary, and pack it all under a budget without ever overflowing.

Everything is deterministic and offline-testable. The default embedder is a pure-stdlib feature-hashing embedder (no embedding API, no numpy), and the summarizer runs behind a provider seam that a ScriptedProvider drives with canned summaries — so the entire test suite and demo run with zero credentials and zero network.

---

Why this exists / what it demonstrates

"RAG" is usually sold as a library call. This repo takes the opposite stance: it makes the context-engineering decisions the point, and implements each one small enough to read in a sitting. Reading it should teach you

• why chunking has knobs — fixed-size windows with overlap (so a fact that straddles a boundary stays retrievable) versus structure-respecting paragraph chunking, both in the same token currency as the budget;
• how retrieval actually ranks — cosine similarity over L2-normalized vectors reduces to a dot product; top-k is exact and deterministic, ties broken by id; and MMR re-ranking trades a little relevance for diversity so you don't spend a scarce budget on five near-duplicates;
• how agents survive long conversations — a rolling summary folds old turns into a dense system note while the recent tail is kept verbatim, and a map-reduce summarizer collapses content larger than a single call can hold;
• the assembly invariants that matter — fill by priority, trim or drop the lowest-priority sections first, never exceed the budget, and never break the recent-turns tail;
• how to make all of this hermetic — a deterministic hashing embedder and a scripted summarizer mean the pipeline is reproducible byte-for-byte and needs no network to test.

---

Architecture

   documents                              conversation history
       │                                          │
       ▼                                          ▼
 ┌───────────┐   chunk.py                 ┌───────────────┐  compact.py
 │  chunk    │  fixed / paragraph         │   compactor   │  rolling summary +
 └─────┬─────┘  (stable ids, overlap)     │ (Provider ▼)  │  map-reduce
       │                                   └───────┬───────┘
       ▼  embed.py                                 │        ┌────────────────┐
 ┌───────────┐  HashingEmbedder (stdlib)           │        │ AnthropicProv  │ Bedrock
 │ embeddings│  deterministic, L2-normalized       └───────▶│   (or)         │ / direct
 └─────┬─────┘                                               │ ScriptedProv   │ offline
       ▼  store.py                                           └────────────────┘
 ┌───────────┐  InMemory / Jsonl              rolling summary  +  recent tail
 │  vector   │  cosine top-k, filter, tie-break        │              │
 │   store   │                                         ▼              ▼
 └─────┬─────┘        retrieve.py           ┌───────────────────────────────┐
       │      ┌───────────────┐  memory     │        assemble.py            │
 query ┴─────▶│   Retriever   │────────────▶│      ContextAssembler         │
              │  plain / MMR  │             │  fill by priority under a     │
              └───────────────┘             │  token budget; trim/drop      │
                                            │  lowest first; keep the tail  │
                budget.py  ◀────────────────┤  ── never exceed the budget   │
                chars/4 estimate + trim     └───────────────┬───────────────┘
                                                            ▼
                                              final ordered context (Messages)

The provider boundary is the only seam that can touch a network, and it is used for exactly one thing — summarization during compaction. AnthropicBedrock and Anthropic expose the identical .messages.create(...) surface, so swapping Bedrock for a direct API key is a one-line change; ScriptedProvider slots into the same seam to keep everything offline.

---

Module map

Module	Responsibility
loom/types.py	Frozen value types the pipeline speaks: Document, Chunk, ScoredChunk, Message, Usage. Pure data, no behavior.
loom/provider.py	Provider protocol + AnthropicProvider (the only file importing anthropic, lazily) and ScriptedProvider. Used solely for summarization.
loom/embed.py	Embedder protocol + HashingEmbedder: deterministic, pure-stdlib feature hashing, L2-normalized. The default embedder; seam for a real API left explicit.
loom/vector.py	Dependency-free vector math: dot, norm, cosine, l2_normalize.
loom/chunk.py	fixed_size_chunks (token-aware sliding window with overlap) and paragraph_chunks (structure-aware, sentence fallback). Stable chunk ids.
loom/store.py	VectorStore protocol, InMemoryVectorStore (exact cosine top-k, metadata filter, id tie-break), JsonlVectorStore (persist/reload, stdlib json).
loom/retrieve.py	Retriever: embed a query, search, and re-rank for diversity via retrieve_mmr (Maximal Marginal Relevance).
loom/compact.py	Compactor: rolling summary + recent window, and recursive map-reduce summarization. Reports tokens before/after.
loom/budget.py	chars/4 token estimation and word-aware trimming to a budget.
loom/assemble.py	ContextAssembler + assemble_default: priority-ordered, budget-bounded context assembly with a full accounting of what was kept/trimmed/dropped.
loom/cli.py	python -m loom demo (offline) and demo --live (Bedrock summarization).

---

Quickstart

Uses uv. Requires Python 3.11+.

uv sync --extra dev

1. Offline demo (no credentials, no network)

The fastest way to see the whole pipeline. It ingests a small in-repo corpus, embeds it with the pure-stdlib HashingEmbedder, retrieves both plainly and with MMR, compacts a fabricated long history with a ScriptedProvider, and assembles a budgeted context — all printed, with zero API calls.

uv run loom demo
# or, equivalently:
uv run python examples/context_demo.py

The same flow in code:

from loom import (
    Document, HashingEmbedder, InMemoryVectorStore, Retriever,
    Compactor, ScriptedProvider, assemble_default, Message, Role,
)
from loom.chunk import paragraph_chunks

# ingest + embed
embedder = HashingEmbedder(dim=256)
store = InMemoryVectorStore()
doc = Document(id="vectors", text="Cosine similarity of normalized vectors is a dot product.")
chunks = paragraph_chunks(doc, size=64)
store.add([c.with_embedding(v) for c, v in zip(chunks, embedder.embed([c.text for c in chunks]))])

# retrieve (plain + diverse)
retriever = Retriever(embedder, store)
hits = retriever.retrieve("cosine similarity", k=3)
diverse = retriever.retrieve_mmr("cosine similarity", k=3, lambda_=0.5)

# compact a long history offline
compactor = Compactor(ScriptedProvider(summaries=["dense summary of older turns"]))
compacted = compactor.rolling_compact(history, keep_recent=2)

# assemble a budgeted context that never overflows
result = assemble_default(
    budget=120,
    system="You are a helpful assistant.",
    memory="\n".join(h.chunk.text for h in hits),
    summary=compacted.messages[0].content,
    recent=[Message(role=Role.USER, content="what did we decide?")],
)
assert result.total_tokens <= result.budget   # the load-bearing invariant

2. Live summarization against AWS Bedrock

The only change from the offline path is swapping the compaction provider — everything else (chunking, embedding, retrieval, assembly) stays local.

export LOOM_PROVIDER=bedrock          # default; use "anthropic" for the direct API
export AWS_REGION=us-east-1
export LOOM_MODEL=claude-opus-4-8     # Bedrock id resolves to anthropic.claude-opus-4-8
uv run loom demo --live
# or:
uv run python examples/live_bedrock.py

Variable	Default (bedrock)	Meaning
LOOM_PROVIDER	bedrock	bedrock or anthropic
LOOM_MODEL	anthropic.claude-opus-4-8	model id (drop the prefix for anthropic)
AWS_REGION	us-east-1	Bedrock region

The request surface is kept minimal for Opus 4.8: only model, max_tokens, system, and messages are sent — no temperature/top_p/top_k.

---

Development

uv run ruff check .
uv run pytest

Both run fully offline. The suite is hermetic: the deterministic HashingEmbedder and the ScriptedProvider mean nothing hits the network and no credentials are needed. import loom works with anthropic uninstalled.

---

Intentional non-goals

This is a tasteful mini-implementation, scoped deliberately. What's left out is left out on purpose:

• No hosted embedding model. The default HashingEmbedder captures lexical overlap only — it has no learned semantics. That is the right trade for a repo whose point is reproducibility: it is deterministic, offline, and dependency- free. A real embedding API slots in behind the Embedder protocol; the seam is marked in embed.py, and we deliberately do not ship one.
• Exact search, not ANN. The stores do brute-force cosine top-k. Approximate nearest-neighbor indexing (HNSW, IVF) is a scaling concern, not a correctness one; for readable code and assertable rankings, exact search is the right call.
• chars/4 token estimation. Budgeting uses a rough heuristic rather than a real BPE tokenizer. It is intentionally conservative (rounds up) so assembly under-fills rather than overflows the true window. It is not a billing counter.
• Summarization only from the model. The provider seam exists for exactly one job — compaction. loom does not call a model to embed, rank, or assemble; those stay deterministic and offline by design.
• No persistence beyond JSONL. JsonlVectorStore is append-only, stdlib-json, single-file. No database, no deletes/updates — a store is cheap to rebuild, which is the reproducible thing to do.
• Synchronous, single-process. No async, no parallel embedding. The pipeline is sequential and easy to follow; parallelism is an optimization that would add surface area without changing what the repo demonstrates.

Each of these is a place a production system does more — and where this repo deliberately stops, so the core stays legible.

---

The platform

loom is one repo in a five-part agent platform. Each owns a single concern, stands alone, and shares the same spine: a normalized, Bedrock-default provider seam and deterministic, fully-offline tests.

Repo	Concern
cogs	the agent runtime — the loop, tool protocol, provider seam, record/replay
bulkhead	reliable serving — a gateway (retries, circuit breaking, rate limits, caching, failover, budgets) in front of any provider
loom	context engineering — retrieve, compact, and assemble what goes in the window ← this repo
sonar	observability — reconstruct a run as a cost/latency timeline
gauntlet	evaluation — hermetic tool-use tasks scored with pass@k + confidence intervals

How work flows through them:

loom ──assemble context──▶ cogs ──model calls──▶ bulkhead ──▶ provider
                            │
              run cassette ─┴──▶ sonar (timeline, cost)
              eval result ─────▶ gauntlet (pass@k)

The seams are real, not aspirational: loom emits the same normalized Message types cogs runs on, so an assembled context feeds straight into the agent loop; cogs, bulkhead, and loom all share the provider seam; and sonar ingests cogs cassettes and gauntlet results directly. The sonar README has the one-command combined demo.

---

License

MIT © 2026 Deepak