Support docu versioning: changed detection of unstructured data

[vinod0m]

Description of the new feature

The parser should be able to detect and handle the different versions of the same PDF document to generate high-quality metadata for your Retrieval Augmented Generation (RAG) system. By extracting structured data from unstructured documents, we can filter results more effectively and drastically improve retrieval accuracy.

Proposed technical implementation details

refer this video (link). This example is based on LangExtract with Gemini and Ollma models if required. Refer to this link and image for more details.

[onestardao]

It sounds like your proposal is aiming to improve RAG accuracy by handling multiple versions of the same document more intelligently.
From the failure patterns we’ve mapped, this is closely related to SoftwareDevLabs/frontend#2 — Semantic ≠ Embedding (where different document versions or formats cause mismatches between semantic meaning and retrieved chunks).
If you’d like, I can share a detailed breakdown of the issue and tested strategies we’ve used to help other dev teams resolve similar versioning challenges.

[vinod0m]

Yes, you are right!
It would be great if you could share the details. it would be of great help to us

[onestardao]

Sure here’s the detailed breakdown with reproducible fixes:
https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

It includes step-by-step strategies we’ve tested to keep semantic meaning aligned across multiple document versions, plus example setups from other dev teams that resolved similar versioning drift.

it's semantic firewall, it built from math, so no need to chagne infra. MIT License, enjoy it

^_________^

[vinod0m]

@onestardao thanks for providing the link. We will look into it and will try to deploy your suggested method when we try to tackle the semantic related topic on different document versions.

[onestardao]

yes, if any Q let me know