Skip to content

Graphrag local search vs global search vs normal RAG #1877

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

majidsh97
Copy link

Description
This pull request introduces a new Jupyter Notebook example that demonstrates how to compare the results from standard RAG, GraphRAG with local search, and GraphRAG with global search.

The goal is to offer users a clear, hands-on way to evaluate the effectiveness of different GraphRAG search strategies against a baseline RAG implementation for their own datasets and questions.

Proposed Changes
Input: The notebook takes a path to the root directory and some questions for comparison.
it generates answers for the provided questions using three distinct methods:
Evaluation: The generated answers from all three methods are then passed to a Language Model (LLM) which is prompted to provide ratings based on completeness, directness, empowerment, and diversity.
Output: The notebook produces:
Four Parquet tables containing the questions, the generated answers from each method, and their corresponding LLM-generated scores.
A plot that visually summarizes the scores, allowing for an easy comparison of the performance across the different RAG methods.

image

[Yes] I have tested these changes locally.
[Yes] I have reviewed the code changes.
[Yes] I have updated the documentation (if necessary, e.g., adding a mention of the new example).
[No] I have added appropriate unit tests (if applicable, though less common for example notebooks).

@majidsh97 majidsh97 requested review from a team as code owners April 12, 2025 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant