What's New:Smart Heading Recognition Feature For Word Document
Starting from v1.5.5, LightRAG's built-in DOCX parser introduces the Smart Heading feature -- enabling better section-based chunking even for poorly formatted DOCX documents.
Many DOCX documents have visually clear section structures but lack properly configured Word outline levels, or retain incorrect heading styles from editing, copying, and concatenation processes. In the past, these issues could cause section titles to be treated as body text, or regular text to be misidentified as headings, resulting in chunking boundaries that deviate from the document's true semantic sections. For P chunking strategies that rely on heading structures, this creates fragmented context or mixed content from different sections, affecting subsequent indexing and retrieval.
Smart Heading targets such poorly formatted but semantically clear documents, restoring their true heading structures as much as possible:
- Recover Missing Section Titles: Identify content that visually appears as titles but lacks properly configured outline levels, reducing loss of section structures.
- Reduce False Positives of Body Text as Headings: Restore regular body text, complete clauses, captions, dates, and version notes that incorrectly carry heading styles back to body text, avoiding meaningless chunking boundaries.
- Improve Inconsistent Heading Hierarchies: Reduce issues such as inconsistent sibling heading levels, inverted parent-child hierarchies, and skipped levels, making section structures more coherent.
- Distinguish Multiple Contents in Merged Documents: Identify main titles and document boundaries of each piece of content, preventing section level interference between different documents.
- Reduce Noise from Long Tables of Contents: Compress excessively long table of contents, minimizing the interference of duplicate information on indexing and retrieval.
By providing more reliable section structures for subsequent chunking, Smart Heading helps maintain semantic associations between titles and their corresponding body text, making chunking results align better with the document's original content organization.
Smart Heading is an optional capability of the Native DOCX parser and requires activation to take effect. For enabling and configuration instructions, please refer to: docs/FileProcessingPipeline / 中文版.
What's Changed
- fix(webui): restore edges after a drag that ends without camera animation by @danielaskdd in #3325
- fix(api): warn when the server is network-exposed without authentication (GHSA-mmg5-8x8q-v934) by @danielaskdd in #3328
- fix(external): prevent task ID URL path injection by @VectorPeak in #3324
- 🔒 fix(api): gate /health configuration disclosure behind authentication (#3294) by @danielaskdd in #3329
- 🔒 fix(docker): patch dependency CVEs, run as non-root, pin bookworm base (#3296) by @danielaskdd in #3330
- Fix docs links and environment typos by @Wenjunyun123 in #3333
- docs(webui): document Bun 'bin metadata file' error on WSL by @danielaskdd in #3339
- docs: add LLM timeout troubleshooting to Key Configuration Guide by @danielaskdd in #3345
- docs(api-server): clarify X-API-Key and Authorization are mutually exclusive per request, fixes #2632 by @hata33 in #3358
- perf(operate): tokenize each description once in entity/relation summary map-reduce by @hata33 in #3359
- perf(kg): close Milvus and Qdrant clients in finalize() to release connections by @hata33 in #3360
- chore(deps): add httpx2 test dependency for starlette testclient by @danielaskdd in #3366
- docs(rerank): document qwen3-rerank via cohere binding (flat format) by @danielaskdd in #3368
- test: isolate Bedrock health auth env by @danielaskdd in #3369
- fix(llm): close AsyncOpenAI client in nvidia_openai_embed by @hata33 in #3371
- 📝 docs(contributing): require English for Git commit messages by @danielaskdd in #3390
- feat(parser): opt-in smart heading discovery for native docx (smart_heading engine param) by @danielaskdd in #3364
- fix: preserve cached xlsx formula values by @eteriall in #3375
- fix: handle docling JSON result envelopes by @ekkoitac in #3344
New Contributors
- @Wenjunyun123 made their first contribution in #3333
- @hata33 made their first contribution in #3358
- @eteriall made their first contribution in #3375
- @ekkoitac made their first contribution in #3344
Full Changelog: v1.5.4...v1.5.5rc1
