Skip to content

v0.2.18

Latest

Choose a tag to compare

@rejojer rejojer released this 16 Sep 13:31
· 1 commit to main since this release
ac0eed7
  • The PageIndex SDK, local or cloud β€” vectorless, reasoning-based RAG, end to end.
  • Much faster indexing β€” the PageIndex Flash engine gets the tree from layout stats: no LLM involved for the structure generation itself, LLMs only write the node summaries, and tree expansion proposes a wave of nodes concurrently.
client = PageIndexClient()
client.submit_document("report.pdf")
client.chat("What does the report conclude?")

Index, to chat, to agent integration, one client.
Local mode needs no server, no vector DB, no PageIndex API key.

Highlights

  • Flash engine: the local default (mode="standard" keeps the classic LLM pipeline). Embedded bookmarks are consumed when trustworthy, and tree optimization is on by default β€” optimize="merge" for the deterministic LLM-free pass, "full" (default) adds LLM expand, which runs a wave of nodes concurrently instead of one round-trip at a time.
  • One complete surface, local and cloud: PageIndexLocalClient(storage_path=...) is the same client as cloud β€” submit, tree, page content, chat, and agent tools all present in both modes, so code moves between them unchanged.
  • Cloud documents, your own model: api_key decides where your documents live; a configured chat model decides who answers β€” and the two combine. PageIndexClient(api_key="pi-...", chat_model="openai/gpt-5.2") runs the same in-process document-QA engine over the live cloud tool set. Page content flows through your process to your provider on your credentials; doc_id targets at the prompt level; enable_citations stays with the managed chat.
  • Agent integration: the cloud MCP tool contract, in-process β€” client.agent_tools() (plain functions), as_openai_tools(), as_anthropic_tools(), as_claude_mcp(), plus one-call openai_agent_config() / anthropic_runner_config() / claude_agent_config() bundles and agent_instructions() for the system prompt. Cloud clients get the live server tool set over the MCP bridge (read-only endpoint by default); local clients get the in-process subset with the same schemas and envelopes β€” agent prompts port unchanged.
  • Chat surfaces: chat() β€” question in, answer out, on any backend; chat(stream=True) shows the run as it happens, thinking and tool calls woven into the text, or as typed events via .events; chat(protocol="responses" | "messages") drives the OpenAI Responses or Anthropic Messages API natively with that protocol's own shapes, and chat_completions() keeps the OpenAI-compatible envelope β€” all with doc_id targeting, streaming, honest usage accounting, and prompt-cache continuity across turns. Transcripts append verbatim: a protocol lane's output goes back into the next request unchanged.
  • Model & connection knobs: index_model / chat_model, index_backend / chat_backend (and per-call backend) passed verbatim to each lane β€” LiteLLM-routed providers, keyless OpenAI-compatible servers, Azure/Bedrock/Vertex included.
  • index= / chat= slots: the grouped spelling of the flat arguments β€” a string shorthand or a mapping (index={"model": ..., "storage_path": ...}, chat={"model": ..., "backend": ...}). "cloud" / "local" name a side, an optional mode= cross-checks it, and PageIndexLocalClient / PageIndexCloudClient take the same slots. PageIndexCloudClient() reads PAGEINDEX_API_KEY; a bare PageIndexClient() stays local no matter what the environment holds.
  • Typed config shapes: IndexConfig / CloudIndexConfig / LocalIndexConfig / ChatConfig, with py.typed shipped so your type checker sees them.
  • Nothing fails quietly: unknown keys, mixed sides, empty values and mode/content conflicts refuse at construction with the legal vocabulary in the message; dead credentials or a missing model fail the indexing run instead of storing a document with blank summaries; every cloud error carries its HTTP status.
  • Dependencies: Python >= 3.10; openai-agents in the base install (the chat engine); [anthropic] and [claude] extras for those SDKs.

Also in 0.2.18

  • get_document_id(name) looks up a document's ID by its display name in one API call, useful for resolving citation doc names to IDs.
  • Prompt alignment: the SDK's local citation and discovery prompts now match the chat MCP server β€” the invented "Sources" footer is removed, a missing rephrase-and-retry persistence step is restored, and the SDK-only pagination tutorial is removed from the discovery instructions.

Also in 0.2.17:

  • Citations resolve to page coordinates. client.resolve_citations(answer, doc_id=...) reads the citation tags in a cited answer β€” the <cite doc="…" page="…" block="…"/> tags chat(citations=True) writes and the <doc=…;page=…;block=…> tags the managed chat writes β€” and returns one entry per citation with the document's id; for a block-level citation on a cloud document indexed with beta_headers=["block_reference"], the entry also carries the block's page, bounding box, type and text. Pass doc_id= (what chat(doc_id=...) took) to scope the name lookup, or omit it to resolve against your library.
  • One block, one call. client.get_block(doc_id, block_id) returns a single layout block as the API serves it β€” {doc_id, page, block_id, bbox, block_type, text}, bbox as [x0, y0, x1, y1] in thousandths of the page's width and height (0-1000), origin top-left. Cloud only; local page content has no blocks.
  • Flash indexes every document, in every script. Layout decides the tree, never the script: small documents, slide decks, and Japanese, Arabic, Hebrew, Indic, Thai and mixed-script PDFs that used to be refused or lose their headings now index, and the same layout yields the same headings in every script. When no hierarchy is detected, the tree falls back to one node per page (toc_source="pages"); pages before a late first heading get a Preface node, so every page is reachable from some node; toc_source is on every result (detected, bookmarks, hybrid, pages, unreadable).

Full Changelog: v0.2.17...v0.2.18

What's Changed

  • fix table comparison by @zmtomorrow in #505
  • Update citation granularity from line-level to block-level by @zmtomorrow in #506
  • v0.2.18: flash improvements, prompt alignment, get_document_id by @rejojer in #507

Full Changelog: v0.2.17...v0.2.18