papershelf

papershelf is a repo-local semantic retrieval layer for AI coding agents. It indexes research, specs, notes, docs, and other reference material from .papershelf/docs/ into a local libSQL vector database, then exposes focused search through a CLI and installed agent skill.

When to use papershelf

Coding agents already have read, rg, and find; papershelf is not a replacement for known paths, well-named docs, or exact-keyword lookup. It is useful when agents need repo-local prose guidance but ordinary search would waste context: large or distributed documents, conceptual queries, lexically similar distractors, or task wording that differs from source wording.

Use when:

• an agent needs evidence from repo-local prose before coding or making a design decision
• the corpus is large, distributed, vocabulary-mismatched, or otherwise awkward to navigate with exact search
• answers should cite reproducible source paths, chunks, line ranges, and quotes

Why papershelf

papershelf keeps agent-facing research close to the code it informs.

• Repo-local by default — source documents and generated index data live with the project, not in a separate managed document store.
• Auditable retrieval — search results expose source paths, chunks, line ranges, and quotes so agents can justify conclusions with reproducible evidence.
• Context stays focused — agents retrieve only task-relevant passages on demand instead of loading broad reference material into every prompt.

Table of Contents

• When to use papershelf
• Why papershelf
• Quickstart
  • 1. Install
  • 2. Initialize a repo
  • 3. Add documents
  • 4. Index the corpus
  • 5. Try a search
• How it works
• CLI reference
• Index storage
• Configuration
• Agent interface
  • Working with agents
  • Agent protocol
  • Search JSON contract
  • Citation contract
• Development and evals
• License

Quickstart

1. Install

Install the CLI globally from npm:

npm install -g papershelf

2. Initialize a repo

papershelf init

This creates .papershelf/docs/ for the source corpus and installs the project-local agent skill at .agents/skills/papershelf/SKILL.md.

3. Add documents

Add research or reference documents to .papershelf/docs/, such as papers, books, specs, design docs, reports, notes, or Markdown reference material.

4. Index the corpus

papershelf index

This extracts, chunks, embeds, and stores searchable passages in .papershelf/index/.

5. Try a search

papershelf search "How should this design handle retries?" --json

How it works

papershelf keeps retrieval infrastructure close to the codebase. Source documents live in .papershelf/docs/; generated index data lives in .papershelf/index/ and can be rebuilt.

papershelf index fingerprints source files, skips unchanged documents, removes stale index rows, chunks changed documents, embeds those chunks, and stores them in a local libSQL vector index. The index is derived state, not the source of truth.

papershelf search "<question>" --json runs a two-stage retrieval path: it embeds the question, uses the local vector index to find nearby chunks, reranks those candidates against the original question, and returns the selected passages with document IDs, chunk numbers, scores, and source metadata.

The main tradeoff is intentional scope. papershelf favors repository-local, agent-first retrieval over a central document service or human-facing knowledge app. That keeps setup simple, makes citations reproducible, and gives coding agents a narrow JSON contract, but it also means maintainers own corpus quality and need to re-index after document changes.

CLI reference

papershelf init
papershelf index [--rebuild]
papershelf search "<question>" --json

Run these commands from the repository root after installing the global CLI.

Index storage

papershelf keeps the source corpus and generated search index inside the repo:

• .papershelf/docs/ — user-added research or reference documents, such as papers, books, specs, design docs, reports, notes, or Markdown reference material.
• .papershelf/index/ — generated local libSQL database; safe to rebuild with papershelf index --rebuild.

The index uses libSQL native F32_BLOB vectors with a DiskANN cosine index and fixed 1280-dimensional embeddings. If a future CLI version cannot use an existing generated index schema, run papershelf index --rebuild or delete .papershelf/index/ and run papershelf index.

Configuration

The intended ZeroEntropy integration uses standalone model endpoints only, not managed zsearch document storage:

• ZEROENTROPY_API_KEY
• Optional ZEROENTROPY_BASE_URL for compatible/local test endpoints; defaults to https://api.zeroentropy.dev/v1
• embedding model zembed-1
• embedding dimension 1280
• reranker model zerank-2

Agent interface

This section is the normative interface for agents. Human maintainers set up the corpus and keep it indexed; agents consume it through the installed skill and structured search output.

Working with agents

The primary integration point is the project-local agent skill installed by papershelf init at .agents/skills/papershelf/SKILL.md.

That skill tells agents to search the corpus before answering questions grounded in the corpus or making substantive decisions the corpus may inform. Agents should use focused queries, cite relevant passages, and avoid treating weak or irrelevant results as support for a conclusion.

Agent protocol

papershelf is useful when repository work should be informed by a local research and reference corpus — papers or preprints, but also books, specs, design docs, reports, notes, or Markdown references.

Canonical agent command:

papershelf search "<specific question>" --json

Protocol:

1. Read the installed skill at .agents/skills/papershelf/SKILL.md.
2. Run a focused semantic search before answering questions grounded in the corpus or making substantive coding/design decisions the corpus may inform.
3. Inspect the returned passages; treat them as supporting evidence, not automatic ground truth.
4. If a result is relevant, cite the query, returned docId, chunk, source lines when available, and a short exact quote from text or snippet.
5. If metadata.startLine and metadata.endLine are present, use them to read only the relevant source range when more context is needed.
6. If results are weak, irrelevant, empty, or inconclusive, say so instead of treating them as support.
7. If documents were added or changed, run papershelf index before searching when that is within scope; otherwise ask the maintainer to re-index.

Search JSON contract

Agents should prefer structured output:

papershelf search "How should this design handle retries?" --json

Output shape:

{
  "results": [
    {
      "docId": ".papershelf/docs/example.md",
      "chunkIndex": 3,
      "text": "Full retrieved chunk text...",
      "snippet": "Short normalized preview...",
      "distance": 0.1234,
      "relevanceScore": 0.98,
      "metadata": {
        "heading": "Retry strategy",
        "section": "Architecture > Retry strategy",
        "page": 12,
        "startLine": 40,
        "endLine": 58
      }
    }
  ]
}

Fields:

• docId — repo-relative source path.
• chunkIndex — chunk number within the source document.
• text — full retrieved passage; cite from this when possible.
• snippet — shortened normalized preview.
• distance — vector distance; lower is closer.
• relevanceScore — reranker score when available; higher is better.
• metadata.heading, metadata.section, metadata.page — source location hints when detectable.
• metadata.startLine, metadata.endLine — source line range for targeted reads.

Citation contract

When an answer relies on papershelf evidence, include an Evidence used section:

Evidence used

- query: "exact query string"
- source path: .papershelf/docs/example.md
- chunk: 3
- lines: 40-58
- quote: "short exact quote copied from the returned text or snippet"

Rules:

• Quote only text returned by papershelf search ... --json.
• A later direct read of the source file may provide context, but it does not replace citing the retrieved search result.
• Do not cite weak or irrelevant passages as evidence.
• If the corpus does not support the answer, say so.

Development and evals

papershelf uses ordinary integration tests plus harness-backed evals for retrieval and agent behavior:

pnpm test:unit
pnpm test:integration
pnpm evals
pnpm evals:record

pnpm test:integration runs the CLI lifecycle in temporary repos against a mock ZeroEntropy server. It verifies init, incremental index, search --json, source paths, headings, exact quotes, query embeddings, and reranking behavior.

pnpm evals runs vitest-evals suites. The core retrieval eval builds a temporary .papershelf/docs/ corpus, indexes it, runs real search code, and asserts retrieval quality across paraphrased queries, including mean reciprocal rank and expected snippets from multi-chunk documents.

The eval harnesses return normalized runs: app-facing output, traceable session data, provider/tool calls, usage, artifacts, and serialized errors. That lets tests assert both the answer and the process, for example that the reranker ran or that an agent used the right search command before citing evidence.

Provider calls for the core evals are recorded under .vitest-evals/recordings/, and the eval config runs in strict replay mode by default. Use pnpm evals:record with ZEROENTROPY_API_KEY when intentionally refreshing recordings.

The optional agent compliance eval is gated by PAPERSHELF_RUN_AGENT_EVALS=1. It runs a Pi agent against the installed papershelf skill and asserts trace-level behavior: the agent must call papershelf search ... --json, cite an exact returned query/source/quote, and use returned line metadata for targeted reads in long documents.

License

MIT