fix: ThemesPersonasInput model validation error when generate multi-hop query

[czhiming-maker]

Issue Link / Problem Description

ragas version: 0.3.3

I meet a problem when i try to generate multi-hop query based on some documents.

console log show that the ThemesPersonasInput model validation error:
pydantic_core._pydantic_core.ValidationError: 2 validation errors for ThemesPersonasInput
themes.0
Input should be a valid string [type=string_type, input_value=('车险代理', '车险'), input_type=tuple]
For further information visit https://errors.pydantic.dev/2.11/v/string_type
themes.1
Input should be a valid string [type=string_type, input_value=('交强险', '交强保险'), input_type=tuple]

I also found that has someone already report this issue: #2274

Changes Made

I found that ThemesPersonasInput model define the "themes" field as List[str], but actually themes value is List[tuple[str, str]], so this error occurred.

I changed the "themes" field to List[t.Union[str, t.Tuple[str, str]]] to solve this error.
because I find that when themes value generated, it always be tuple list, code below show the reason:
https://github.com/explodinggradients/ragas/blob/main/src/ragas/testset/transforms/relationship_builders/traditional.py#L162

Testing

Manual testing steps:

1. change code as described above
2. try to generate multi-hop query again
3. the result show success

Test code:

from ragas.llms import LangchainLLMWrapper
from ragas.embeddings import LangchainEmbeddingsWrapper
from ragas.testset import TestsetGenerator
from langchain_community.document_loaders import JSONLoader
from langchain_google_genai import GoogleGenerativeAIEmbeddings, ChatGoogleGenerativeAI
from ragas.testset.synthesizers.multi_hop.specific import (
    MultiHopSpecificQuerySynthesizer,
)

import os
os.environ["GOOGLE_API_KEY"] = "xxxx-O5-xxxxx"

loader = JSONLoader(
    file_path="data/documents.json",
    jq_schema=".[].content",
)
documents = loader.load()

generator_llm = LangchainLLMWrapper(ChatGoogleGenerativeAI(model="gemini-2.5-flash"))
generator_embeddings = LangchainEmbeddingsWrapper(GoogleGenerativeAIEmbeddings(
    model="models/gemini-embedding-001",
))

distribution = [
    (MultiHopSpecificQuerySynthesizer(llm=generator_llm), 1),
]
generator = TestsetGenerator(llm=generator_llm, embedding_model=generator_embeddings)

async def generate():
    # generate testset
    testset = generator.generate_with_langchain_docs(
        documents,
        testset_size=4, 
        query_distribution=distribution,
    )

    testset.to_evaluation_dataset().to_jsonl("testset.jsonl")

import asyncio
asyncio.run(generate())

References

• Related issues: Unable to create Test data with TestsetGenerator #2274

[anistark]

Thanks for the PR @czhiming-maker

This might break the definitions and type checkers.

Have you considered updating traditional.py to generate only strings instead of tuples?

Or can also do explicit modeling for it. Open for discussion if needed.

[czhiming-maker]

Thanks for the PR @czhiming-maker

This might break the definitions and type checkers.

Have you considered updating traditional.py to generate only strings instead of tuples?

Or can also do explicit modeling for it. Open for discussion if needed.

Good idea.
I've decided not to change the original schema's structure definition. I'll replace the assignment of overlapped_items in traditional.py from tuples to strings.
Thanks.

[anistark]

Good idea. I've decided not to change the original schema's structure definition. I'll replace the assignment of overlapped_items in traditional.py from tuples to strings. Thanks.

Great. Let us know how it goes. Closing this PR for the time being. Feel free to open a new one if there's any other changes you'd want to add. :)

[czhiming-maker]

Good idea. I've decided not to change the original schema's structure definition. I'll replace the assignment of overlapped_items in traditional.py from tuples to strings. Thanks.

Great. Let us know how it goes. Closing this PR for the time being. Feel free to open a new one if there's any other changes you'd want to add. :)

OK，I have opened a new PR about this, #2313
Thank You.