feat: compile pipeline, query agent, and multimodal improvements

[rejojer]

Summary

• Concept dedup & compile refactor: concept plan/update/related paths, bidirectional backlinks, shared _compile_concepts
• Brief system & unified query: per-page JSON sources via pymupdf, doc_type + full_text frontmatter, get_page_content tool
• Config & environment: default model gpt-5.4-mini, API key warning, warning suppression, test isolation
• CLI polish: improved init prompts with explicit defaults, American English, output formatting
• Multimodal query: get_image tool with ToolOutputImage for viewing figures/charts in source documents
• Image path unification: all image paths use sources/images/ prefix, pymupdf replaces PageIndex for page content extraction

Test plan

• All 184 tests pass
• openkb init shows correct prompts with defaults
• openkb add short doc: clean frontmatter, images at sources/images/
• openkb add long doc: per-page JSON from pymupdf, correct image paths
• openkb query on short doc: reads source via read_file
• openkb query on long doc: uses get_page_content with targeted pages
• openkb query about figures: calls get_image tool

[rejojer]

Code review

Found 1 issue:

1. _write_concept appends LLM-rewritten content instead of replacing the body during concept updates, causing content duplication. The _CONCEPT_UPDATE_USER prompt (line 106) instructs the LLM to "Rewrite the full page incorporating the new information naturally -- do not just append" and return "The rewritten full concept page." However, _write_concept appends the complete rewrite to the existing body (existing += f"\n\n{clean}" at line 341) rather than replacing the body with it. Every concept update with a new source document will duplicate all existing content.

OpenKB/openkb/agent/compiler.py

Lines 319 to 342 in 44bf83e

	if is_update and path.exists():
	existing = path.read_text(encoding="utf-8")
	if source_file not in existing:
	if existing.startswith("---"):
	end = existing.find("---", 3)
	if end != -1:
	fm = existing[:end + 3]
	body = existing[end + 3:]
	if "sources:" in fm:
	fm = fm.replace("sources: [", f"sources: [{source_file}, ")
	else:
	fm = fm.replace("---\n", f"---\nsources: [{source_file}]\n", 1)
	existing = fm + body
	else:
	existing = f"---\nsources: [{source_file}]\n---\n\n" + existing
	# Strip frontmatter from LLM content to avoid duplicate blocks
	clean = content
	if clean.startswith("---"):
	end = clean.find("---", 3)
	if end != -1:
	clean = clean[end + 3:].lstrip("\n")
	existing += f"\n\n{clean}"
	if brief and existing.startswith("---"):

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[rejojer]

Code review

Found 1 issue:

1. _write_concept silently discards LLM-generated content on re-compilation of the same document. When is_update=True and the concept page already exists, the body-replacement logic (lines 339-353) is entirely nested inside if source_file not in existing:. On re-compile, source_file is already present in the frontmatter, so this condition is false and the LLM rewrite is silently dropped -- the API call is made and paid for, but the result is never written. Only the brief field gets updated (lines 354-363, outside the gate). The source_file not in existing check should only guard the "add source to frontmatter" block, not the body replacement.

OpenKB/openkb/agent/compiler.py

Lines 323 to 354 in 3dd84f3

	if is_update and path.exists():
	existing = path.read_text(encoding="utf-8")
	if source_file not in existing:
	if existing.startswith("---"):
	end = existing.find("---", 3)
	if end != -1:
	fm = existing[:end + 3]
	body = existing[end + 3:]
	if "sources:" in fm:
	fm = fm.replace("sources: [", f"sources: [{source_file}, ")
	else:
	fm = fm.replace("---\n", f"---\nsources: [{source_file}]\n", 1)
	existing = fm + body
	else:
	existing = f"---\nsources: [{source_file}]\n---\n\n" + existing
	# Strip frontmatter from LLM content to avoid duplicate blocks
	clean = content
	if clean.startswith("---"):
	end = clean.find("---", 3)
	if end != -1:
	clean = clean[end + 3:].lstrip("\n")
	# Replace body with LLM rewrite (prompt asks for full rewrite, not delta)
	if existing.startswith("---"):
	end = existing.find("---", 3)
	if end != -1:
	existing = existing[:end + 3] + "\n\n" + clean
	else:
	existing = clean
	else:
	existing = clean
	if brief and existing.startswith("---"):

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[rejojer]

Code review

Found 1 issue:

1. _write_concept silently discards LLM-generated concept rewrites on re-compilation. When is_update=True and source_file already appears in the existing concept page (in sources: [...] frontmatter), the source_file not in existing guard at line 326 causes the entire body-replacement block (lines 327-353) to be skipped. Only the brief frontmatter field is updated; the full page rewrite from _CONCEPT_UPDATE_USER is thrown away. This means re-adding a modified document leaves concept pages stale. (bug: body replacement wrongly gated inside source-dedup check)

OpenKB/openkb/agent/compiler.py

Lines 323 to 354 in ef60f7d

	if is_update and path.exists():
	existing = path.read_text(encoding="utf-8")
	if source_file not in existing:
	if existing.startswith("---"):
	end = existing.find("---", 3)
	if end != -1:
	fm = existing[:end + 3]
	body = existing[end + 3:]
	if "sources:" in fm:
	fm = fm.replace("sources: [", f"sources: [{source_file}, ")
	else:
	fm = fm.replace("---\n", f"---\nsources: [{source_file}]\n", 1)
	existing = fm + body
	else:
	existing = f"---\nsources: [{source_file}]\n---\n\n" + existing
	# Strip frontmatter from LLM content to avoid duplicate blocks
	clean = content
	if clean.startswith("---"):
	end = clean.find("---", 3)
	if end != -1:
	clean = clean[end + 3:].lstrip("\n")
	# Replace body with LLM rewrite (prompt asks for full rewrite, not delta)
	if existing.startswith("---"):
	end = existing.find("---", 3)
	if end != -1:
	existing = existing[:end + 3] + "\n\n" + clean
	else:
	existing = clean
	else:
	existing = clean
	if brief and existing.startswith("---"):

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}