Skip to content

v1.0.0 - Synthadoc Community Edition

Latest
Latest

Choose a tag to compare

View all tags
@william-Johnason william-Johnason released this 04 Jul 22:48
v1.0.0
cb9eb74
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Synthadoc Community Edition v1.0 Release Notes

Synthadoc Community Edition v1.0 is released.

Five features focused on source quality, retrieval accuracy, and knowledge structure. The foundation work that makes a wiki trustworthy at scale.

1. Pre-LLM Source Sanitizer

Every source document is now cleaned before any LLM call. The sanitizer removes six categories of content that degrade synthesis quality or pose prompt-injection risk, at zero LLM cost.

What is removed:

  • Zero-width characters and Unicode bidi overrides
  • HTML comments and hidden CSS spans
  • Base64 blobs ≥ 200 characters (embedded binary payloads)
  • Instruction-override phrases (prompt injection patterns)

A WARN log entry is emitted when bidi overrides or instruction-override phrases are stripped, so you can trace which sources triggered the filter. Sanitization runs before truncation, so the character budget is always consumed by clean text.

→ Full reference: §4 — Ingest Pipeline

2. Knowledge Graph

Every lint run now builds a wikilink graph across all active and draft pages. Nodes are enriched with page type, lifecycle state, and cluster membership; edges are weighted by reference count. Community detection (Louvain) groups pages into topical clusters automatically.

The graph is browsable from the web UI Graph tab — a D3.js force-directed canvas where nodes are colored by cluster. Click any node to see its title, type, state, and an "Ask about this →" button that opens a pre-filled chat query.

# Graph is built automatically during every lint run
synthadoc lint run

# Access via REST
GET /graph
# Returns: { node_count, edge_count, cluster_count, nodes: [...], edges: [...] }
# First call after upgrade triggers a background build and returns { "status": "computing" }

→ Full reference: Step 24 — Knowledge Graph

3. Per-Source Truncation Flag

The default per-source character limit has been raised from 8,000 to 32,000 characters (~8,000 tokens). Sources that exceed the limit are still ingested — their content is truncated cleanly — and the page is flagged so you know which pages may have incomplete coverage.

# Override the limit per run
synthadoc ingest report.pdf --max-source-chars 64000

# Lint warns on any page with a truncated source
synthadoc lint report
# [WARN] page "q3-earnings" has 1 truncated source — consider --max-source-chars 64000

Frontmatter flag on truncated pages:

sources:
  - file: q3-earnings.pdf
    truncated: true

Config key ([ingest] max_source_chars, default 32000) sets the limit globally. The same override is available via POST /jobs/ingest body field and the MCP synthadoc_ingest parameter.

Full reference: §30 — Per-Source Truncation Flag

4. Proportional Context Budget

Query context is now allocated proportionally to the configured model's context window, rather than a fixed page count. Four configurable slices control how the window is divided:

Slice Key Default
Wiki pages context_wiki_pct 60%
Conversation history context_history_pct 20%
System prompt context_system_pct 15%
Wiki index context_index_pct 5%

A built-in model table covers Claude 4, GPT-4o/turbo/3.5, and GPT-4 variants. Unknown or local models fall back to 128,000 tokens, or you can set an explicit override:

# .synthadoc/config.toml
[query]
context_window = 32000        # override for local/custom models
context_wiki_pct = 0.65       # optional: tune individual slices

5. Page-Level Citation Hardening

Six correctness fixes to the ingest citation pipeline, making citation markers more accurate and more consistently rendered:

  • Full source text to decision LLM : citations can now reference any line in the document, not just the summary window
  • _annotate_citations bugs A–F resolved : false-positive matches, off-by-one line ranges, duplicate markers, escaped bracket handling, multi-paragraph span errors, and source filename normalisation
  • _has_citations lint check : lint now warns when a page has zero citation markers after synthesis and suggests a model upgrade if the pattern persists
  • Key Data section hardening : numerical facts, rates, and formulas are preserved verbatim across the annotation pass
  • Web UI citation rendering fixed : citation superscripts now display correctly in the chat panel alongside streamed answers
  • Reading view default : Obsidian now opens wiki pages in Reading View by default, where citation chips are interactive

→ Full reference: Step 6 : Ingest and citations

Also in this release: AquaFlow Capital - Finance Domain Example

A worked example showing Synthadoc applied to a realistic private equity due diligence scenario is included in docs/example/aquaflow/.

The scenario: AquaFlow Capital is evaluating an LBO of a mid-market water treatment company. Eight deal documents are provided as raw sources - company profile, sector analysis, LBO model mechanics, covenant framework, quality of earnings guide, ESG standards, legal DD process, and exit valuation benchmarks. The walkthrough takes you from synthadoc install through batch ingest, scaffold, lint, and natural-language query.

It is a useful reference if you are building a wiki for a domain with structured financial or analytical source material, want to see how the ingest agent handles heterogeneous document types in a single batch, or want a concrete example of the full growth loop before starting your own domain.

→ Full walkthrough: docs/example/aquaflow/README.md

Full changelog: [v0.9.3...v1.0.0]

Assets 2
Loading