Repository navigation
File Metadata and Filtering #245
Replies: 2 comments
|
I had a quick look at the current public surface. I would separate two levels here. Today I can see document identity and citation metadata ( For a production RBAC use case, I would be careful not to treat metadata passed to the agent as the access-control boundary. The safer shape is:
For RAG-Anything specifically, the minimal feature I would look for is a single So my reading is: |
|
Hey! This is an interesting use case, especially for production RAG systems with multi-tenant access control. I've been working with local RAG systems, and I would approach this in two parts: 1. Metadata storage and filtering Based on the current LightRAG API, I don't see a first-class For your Qdrant setup, ideally, each chunk should carry payload metadata such as: {
"file_path": "/contracts/agreement.pdf",
"tenant_id": "tenant_123",
"allowed_roles": ["admin", "legal"]
}Then you can apply Qdrant payload filters before returning retrieval results. 2. RBAC and security One important thing: I wouldn't rely on filtering retrieved content through the LLM prompt. Unauthorized documents should never enter the model's context in the first place. The authorization filter should be applied at the retrieval layer and consistently enforced across both vector retrieval and graph-based retrieval. Until native filtering is available, separate LightRAG instances or storage namespaces for different security boundaries could be a practical workaround. Otherwise, implementing custom metadata propagation and query filtering in the underlying LightRAG retrieval pipeline may be necessary. I also found an upstream LightRAG discussion proposing metadata filtering: It would be great to see native metadata filtering supported in RAG-Anything, especially for production multi-tenant applications. |
Uh oh!
There was an error while loading. Please reload this page.
Hi
I currently have a Qdrant vector database in production. I am thinking about switching over to RAG-Anything. I am curious if RAG-Anything has support for attaching metadata to Files/chunks/nodes at ingestion time. The reason for these metadata fields are to allow the search to be filtered. These metadata Key Value (KV's) pairs would allow us to restrict data to what users are allowed to see and also provide the agent with more information about the file. For instance, in Qdrant I attach roles and file_paths to the metadata, so that I can control file access via RBAC and then also provide the file_path to my agent so it can directly pull files from our virtual file system (VFS) via VFS tools.
Does anyone have insight into if this is currently supported or if it is planned to be supported? It would need to be supported for both the vector database and graph nodes.
All reactions