[Bug]: The language is specified in the prompt tune, but the content of the community report is in English.

[jjkings]

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
• I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the bug

The content of the community report in the GraphRAG index is always in English.
In GraphRAG 1.1.1, the community report was generated in the specified language by setting the language in the prompt tune.
However, in version 1.2, the language setting is ignored and the report is always output in English.

The issue seems to be caused by the prompt used for generating the community report via prompt tuning.
In GraphRAG 1.1.1, the template for this prompt included the instruction "Everything should be in {language}." to enforce language-specific output.

graphrag/graphrag/prompt_tune/template/community_report_summarization.py

Lines 6 to 95 in 2682c71

	COMMUNITY_REPORT_SUMMARIZATION_PROMPT = """
	{persona}
	# Goal
	Write a comprehensive assessment report of a community taking on the role of a {role}. The content of this report includes an overview of the community's key entities and relationships.
	# Report Structure
	The report should include the following sections:
	- TITLE: community's name that represents its key entities - title should be short but specific. When possible, include representative named entities in the title.
	- SUMMARY: An executive summary of the community's overall structure, how its entities are related to each other, and significant points associated with its entities.
	- REPORT RATING: {report_rating_description}
	- RATING EXPLANATION: Give a single sentence explanation of the rating.
	- DETAILED FINDINGS: A list of 5-10 key insights about the community. Each insight should have a short summary followed by multiple paragraphs of explanatory text grounded according to the grounding rules below. Be comprehensive.
	Return output as a well-formed JSON-formatted string with the following format. Don't use any unnecessary escape sequences. The output should be a single JSON object that can be parsed by json.loads.
	{{
	"title": "<report_title>",
	"summary": "<executive_summary>",
	"rating": <threat_severity_rating>,
	"rating_explanation": "<rating_explanation>"
	"findings": "[{{"summary":"<insight_1_summary>", "explanation": "<insight_1_explanation"}}, {{"summary":"<insight_2_summary>", "explanation": "<insight_2_explanation"}}]"
	}}
	# Grounding Rules
	After each paragraph, add data record reference if the content of the paragraph was derived from one or more data records. Reference is in the format of [records: <record_source> (<record_id_list>, ...<record_source> (<record_id_list>)]. If there are more than 10 data records, show the top 10 most relevant records.
	Each paragraph should contain multiple sentences of explanation and concrete examples with specific named entities. All paragraphs must have these references at the start and end. Use "NONE" if there are no related roles or records. Everything should be in {language}.
	Example paragraph with references added:
	This is a paragraph of the output text [records: Entities (1, 2, 3), Claims (2, 5), Relationships (10, 12)]
	# Example Input
	-----------
	Text:
	Entities
	id,entity,description
	5,ABILA CITY PARK,Abila City Park is the location of the POK rally
	Relationships
	id,source,target,description
	37,ABILA CITY PARK,POK RALLY,Abila City Park is the location of the POK rally
	38,ABILA CITY PARK,POK,POK is holding a rally in Abila City Park
	39,ABILA CITY PARK,POKRALLY,The POKRally is taking place at Abila City Park
	40,ABILA CITY PARK,CENTRAL BULLETIN,Central Bulletin is reporting on the POK rally taking place in Abila City Park
	Output:
	{{
	"title": "Abila City Park and POK Rally",
	"summary": "The community revolves around the Abila City Park, which is the location of the POK rally. The park has relationships with POK, POKRALLY, and Central Bulletin, all
	of which are associated with the rally event.",
	"rating": 5.0,
	"rating_explanation": "The impact rating is moderate due to the potential for unrest or conflict during the POK rally.",
	"findings": [
	{{
	"summary": "Abila City Park as the central location",
	"explanation": "Abila City Park is the central entity in this community, serving as the location for the POK rally. This park is the common link between all other
	entities, suggesting its significance in the community. The park's association with the rally could potentially lead to issues such as public disorder or conflict, depending on the
	nature of the rally and the reactions it provokes. [records: Entities (5), Relationships (37, 38, 39, 40)]"
	}},
	{{
	"summary": "POK's role in the community",
	"explanation": "POK is another key entity in this community, being the organizer of the rally at Abila City Park. The nature of POK and its rally could be a potential
	source of threat, depending on their objectives and the reactions they provoke. The relationship between POK and the park is crucial in understanding the dynamics of this community.
	[records: Relationships (38)]"
	}},
	{{
	"summary": "POKRALLY as a significant event",
	"explanation": "The POKRALLY is a significant event taking place at Abila City Park. This event is a key factor in the community's dynamics and could be a potential
	source of threat, depending on the nature of the rally and the reactions it provokes. The relationship between the rally and the park is crucial in understanding the dynamics of this
	community. [records: Relationships (39)]"
	}},
	{{
	"summary": "Role of Central Bulletin",
	"explanation": "Central Bulletin is reporting on the POK rally taking place in Abila City Park. This suggests that the event has attracted media attention, which could
	amplify its impact on the community. The role of Central Bulletin could be significant in shaping public perception of the event and the entities involved. [records: Relationships
	(40)]"
	}}
	]
	}}
	# Real Data
	Use the following text for your answer. Do not make anything up in your answer.
	Text:
	{{input_text}}
	Output:"""

In version 1.2, however, this instruction has been removed.

graphrag/graphrag/prompt_tune/template/community_report_summarization.py

Lines 6 to 104 in 0d363e6

	COMMUNITY_REPORT_SUMMARIZATION_PROMPT = """
	{persona}
	# Goal
	Write a comprehensive assessment report of a community taking on the role of a {role}. The content of this report includes an overview of the community's key entities and relationships.
	# Report Structure
	The report should include the following sections:
	- TITLE: community's name that represents its key entities - title should be short but specific. When possible, include representative named entities in the title.
	- SUMMARY: An executive summary of the community's overall structure, how its entities are related to each other, and significant information associated with its entities.
	- REPORT RATING: {report_rating_description}
	- RATING EXPLANATION: Give a single sentence explanation of the rating.
	- DETAILED FINDINGS: A list of 5-10 key insights about the community. Each insight should have a short summary followed by multiple paragraphs of explanatory text grounded according to the grounding rules below. Be comprehensive.
	Return output as a well-formed JSON-formatted string with the following format. Don't use any unnecessary escape sequences. The output should be a single JSON object that can be parsed by json.loads.
	{{
	"title": <report_title>,
	"summary": <executive_summary>,
	"rating": <impact_severity_rating>,
	"rating_explanation": <rating_explanation>,
	"findings": [
	{{
	"summary":<insight_1_summary>,
	"explanation": <insight_1_explanation>
	}},
	{{
	"summary":<insight_2_summary>,
	"explanation": <insight_2_explanation>
	}}
	]
	}}
	# Grounding Rules
	Points supported by data should list their data references as follows:
	"This is an example sentence supported by multiple data references [Data: <dataset name> (record ids); <dataset name> (record ids)]."
	Do not list more than 5 record ids in a single reference. Instead, list the top 5 most relevant record ids and add "+more" to indicate that there are more.
	For example:
	"Person X is the owner of Company Y and subject to many allegations of wrongdoing [Data: Reports (1), Entities (5, 7); Relationships (23); Claims (7, 2, 34, 64, 46, +more)]."
	where 1, 5, 7, 23, 2, 34, 46, and 64 represent the id (not the index) of the relevant data record.
	Do not include information where the supporting evidence for it is not provided.
	# Example Input
	-----------
	Text:
	Entities
	id,entity,description
	5,VERDANT OASIS PLAZA,Verdant Oasis Plaza is the location of the Unity March
	6,HARMONY ASSEMBLY,Harmony Assembly is an organization that is holding a march at Verdant Oasis Plaza
	Relationships
	id,source,target,description
	37,VERDANT OASIS PLAZA,UNITY MARCH,Verdant Oasis Plaza is the location of the Unity March
	38,VERDANT OASIS PLAZA,HARMONY ASSEMBLY,Harmony Assembly is holding a march at Verdant Oasis Plaza
	39,VERDANT OASIS PLAZA,UNITY MARCH,The Unity March is taking place at Verdant Oasis Plaza
	40,VERDANT OASIS PLAZA,TRIBUNE SPOTLIGHT,Tribune Spotlight is reporting on the Unity march taking place at Verdant Oasis Plaza
	41,VERDANT OASIS PLAZA,BAILEY ASADI,Bailey Asadi is speaking at Verdant Oasis Plaza about the march
	43,HARMONY ASSEMBLY,UNITY MARCH,Harmony Assembly is organizing the Unity March
	Output:
	{{
	"title": "Verdant Oasis Plaza and Unity March",
	"summary": "The community revolves around the Verdant Oasis Plaza, which is the location of the Unity March. The plaza has relationships with the Harmony Assembly, Unity March, and Tribune Spotlight, all of which are associated with the march event.",
	"rating": 5.0,
	"rating_explanation": "The impact severity rating is moderate due to the potential for unrest or conflict during the Unity March.",
	"findings": [
	{{
	"summary": "Verdant Oasis Plaza as the central location",
	"explanation": "Verdant Oasis Plaza is the central entity in this community, serving as the location for the Unity March. This plaza is the common link between all other entities, suggesting its significance in the community. The plaza's association with the march could potentially lead to issues such as public disorder or conflict, depending on the nature of the march and the reactions it provokes. [Data: Entities (5), Relationships (37, 38, 39, 40, 41,+more)]"
	}},
	{{
	"summary": "Harmony Assembly's role in the community",
	"explanation": "Harmony Assembly is another key entity in this community, being the organizer of the march at Verdant Oasis Plaza. The nature of Harmony Assembly and its march could be a potential source of threat, depending on their objectives and the reactions they provoke. The relationship between Harmony Assembly and the plaza is crucial in understanding the dynamics of this community. [Data: Entities(6), Relationships (38, 43)]"
	}},
	{{
	"summary": "Unity March as a significant event",
	"explanation": "The Unity March is a significant event taking place at Verdant Oasis Plaza. This event is a key factor in the community's dynamics and could be a potential source of threat, depending on the nature of the march and the reactions it provokes. The relationship between the march and the plaza is crucial in understanding the dynamics of this community. [Data: Relationships (39)]"
	}},
	{{
	"summary": "Role of Tribune Spotlight",
	"explanation": "Tribune Spotlight is reporting on the Unity March taking place in Verdant Oasis Plaza. This suggests that the event has attracted media attention, which could amplify its impact on the community. The role of Tribune Spotlight could be significant in shaping public perception of the event and the entities involved. [Data: Relationships (40)]"
	}}
	]
	}}
	# Real Data
	Use the following text for your answer. Do not make anything up in your answer.
	Text:
	{{input_text}}
	Output:"""

Steps to reproduce

1. Create GraphRAG index folder and modify the config file settings.yaml.
2. Tuning prompts for the specific language. then copy generated prompts into index folder.

graphrag prompt-tune --root indexroot --language Japanese

3. Copy documents and run the indexer.
4. Load dataframe from the file indexroot\output\community_reports.parquet. Content of the fields "title", "summary", "full_content" are in English.

Expected Behavior

No response

GraphRAG Config Used

### This config file contains required core defaults that must be set, along with a handful of common optional settings.
### For a full list of available settings, see https://microsoft.github.io/graphrag/config/yaml/

### LLM settings ###
## There are a number of settings to tune the threading and token limits for LLM calls - check the docs.

models:
  default_chat_model:
    #type: openai_chat # or azure_openai_chat
    type: azure_openai_chat
    api_base: https://myopenai.openai.azure.com/
    api_version: 2024-05-01-preview
    auth_type: api_key # or azure_managed_identity
    api_key: ${GRAPHRAG_API_KEY} # set this in the generated .env file
    # audience: "https://cognitiveservices.azure.com/.default"
    # organization: <organization_id>
    model: gpt-4o-mini
    deployment_name: gpt-4o-mini
    # encoding_model: cl100k_base # automatically set by tiktoken if left undefined
    model_supports_json: true # recommended if this is available for your model.
    concurrent_requests: 25 # max number of simultaneous LLM requests allowed
    async_mode: threaded # or asyncio
    retry_strategy: native
    max_retries: -1                   # set to -1 for dynamic retry logic (most optimal setting based on server response)
    tokens_per_minute: 0              # set to 0 to disable rate limiting
    requests_per_minute: 0            # set to 0 to disable rate limiting
  default_embedding_model:
    #type: openai_embedding # or azure_openai_embedding
    type: azure_openai_embedding
    api_base: https://myopenai.openai.azure.com/
    api_version: 2024-05-01-preview
    auth_type: api_key # or azure_managed_identity
    api_key: ${GRAPHRAG_API_KEY}
    # audience: "https://cognitiveservices.azure.com/.default"
    # organization: <organization_id>
    model: text-embedding-ada-002
    deployment_name: text-embedding-ada-002
    # encoding_model: cl100k_base # automatically set by tiktoken if left undefined
    model_supports_json: true # recommended if this is available for your model.
    concurrent_requests: 25 # max number of simultaneous LLM requests allowed
    async_mode: threaded # or asyncio
    retry_strategy: native
    max_retries: -1                   # set to -1 for dynamic retry logic (most optimal setting based on server response)
    tokens_per_minute: 0              # set to 0 to disable rate limiting
    requests_per_minute: 0            # set to 0 to disable rate limiting

vector_store:
  default_vector_store:
    type: lancedb
    db_uri: output\lancedb
    container_name: default
    overwrite: True

embed_text:
  model_id: default_embedding_model
  vector_store_id: default_vector_store

### Input settings ###

input:
  type: file # or blob
  file_type: text # [csv, text, json]
  base_dir: "input"

chunks:
  size: 1200
  overlap: 100
  group_by_columns: [id]

### Output settings ###
## If blob storage is specified in the following four sections,
## connection_string and container_name must be provided

cache:
  type: file # [file, blob, cosmosdb]
  base_dir: "cache"

reporting:
  type: file # [file, blob, cosmosdb]
  base_dir: "logs"

output:
  type: file # [file, blob, cosmosdb]
  base_dir: "output"

### Workflow settings ###

extract_graph:
  model_id: default_chat_model
  prompt: "prompts/extract_graph.txt"
  entity_types: [organization,person,geo,event]
  max_gleanings: 1

summarize_descriptions:
  model_id: default_chat_model
  prompt: "prompts/summarize_descriptions.txt"
  max_length: 500

extract_graph_nlp:
  text_analyzer:
    extractor_type: regex_english # [regex_english, syntactic_parser, cfg]

extract_claims:
  enabled: false
  model_id: default_chat_model
  prompt: "prompts/extract_claims.txt"
  description: "Any claims or facts that could be relevant to information discovery."
  max_gleanings: 1

community_reports:
  model_id: default_chat_model
  graph_prompt: "prompts/community_report_graph.txt"
  text_prompt: "prompts/community_report_text.txt"
  max_length: 2000
  max_input_length: 8000

cluster_graph:
  max_cluster_size: 10

embed_graph:
  enabled: false # if true, will generate node2vec embeddings for nodes

umap:
  enabled: false # if true, will generate UMAP embeddings for nodes (embed_graph must also be enabled)

snapshots:
  graphml: false
  embeddings: false

### Query settings ###
## The prompt locations are required here, but each search method has a number of optional knobs that can be tuned.
## See the config docs: https://microsoft.github.io/graphrag/config/yaml/#query

local_search:
  chat_model_id: default_chat_model
  embedding_model_id: default_embedding_model
  prompt: "prompts/local_search_system_prompt.txt"

global_search:
  chat_model_id: default_chat_model
  map_prompt: "prompts/global_search_map_system_prompt.txt"
  reduce_prompt: "prompts/global_search_reduce_system_prompt.txt"
  knowledge_prompt: "prompts/global_search_knowledge_system_prompt.txt"

drift_search:
  chat_model_id: default_chat_model
  embedding_model_id: default_embedding_model
  prompt: "prompts/drift_search_system_prompt.txt"
  reduce_prompt: "prompts/drift_search_reduce_prompt.txt"

basic_search:
  chat_model_id: default_chat_model
  embedding_model_id: default_embedding_model
  prompt: "prompts/basic_search_system_prompt.txt"

Logs and screenshots

No response

Additional Information

• GraphRAG Version: 1.2
• Operating System: Windows 10
• Python Version: 3.11.9
• Related Issues:

[natoverse]

It looks like this was removed from the prompt template erroneously while working on response quality improvements (which were likely evaluated in English). Adding to the backlog to fix.

In the meantime, the prompts are written to your project folder as text files and can be edited before indexing, so you can add in language instructions directly.

[natoverse]

Resolved with 2.2.1 release