This repository contains the official implementation of the paper: "Enhancing Graph-based Retrieval-Augmented Generation via Query-Aware Path Reasoning"
Our method introduces query-aware path reasoning into the graph-based retrieval-augmented generation (RAG) framework, aiming to improve knowledge selection and reasoning by leveraging structured relational paths.
cd QPathRAG
pip install -e .The end-to-end pipeline is organized into multiple steps under the reproduce/ folder.
- Convert your raw .jsonl dataset into a .json file, where each entry contains a list of document segments.
- Utilize the model’s capability to extract structured knowledge from text.
- Transform the segmented documents produced in Step 0 into a knowledge graph representation.
- Generate summaries from groups of document segments.
- Concatenate them as context and prompt a large language model to create 125 diverse questions.
- Use the knowledge graph obtained in Step 1.
- Let the model answer the 125 questions generated in Step 2 based on structured reasoning over the graph.
- Save the results for evaluation.
- Run
python reproduce/batch_eval.pyto evaluate the model’s responses.
- Finally, compute the win rate with:
python reproduce/calc_win_rate.py- Before running, make sure to set your API key if using LLMs:
api_key = "sk-..."- Prepare your own dataset in .jsonl format and follow the pipeline from Step 0 → Step 3 → Evaluation.