adding RAGAS in example folder

[bandish1304]

I'd like to contribute a RAGAS evaluation pipeline example in example section. I am aware that the example section already have RAG pipeline + pincone but this is different.
Dagster's existing RAG + Pinecone example does:

Ingest documents → embed them → store in Pinecone → answer questions
It's about building the retrieval pipeline

A RAGAS evaluation pipeline will do:

It's about measuring how good the RAG pipeline actually is
For example, Dagster's existing example builds the car. RAGAS tells you whether the car actually drives well.

Is this something the team would welcome?

[cmpadden]

Hey @bandish1304 - thanks for expressing your interest in contributing.

The way I see it, the more examples, the better! Would love to see what you come up with.

[bandish1304]

Hi @cmpadden - Thanks for the encouragement! I’ll put together a RAGAS evaluation pipeline example and share a draft PR soon for feedback

[musaabhasan]

I think this would be valuable if the example is framed around orchestration and evidence, not just calling RAGAS inside a script.

What would make it feel Dagster-native:

• assets for the evaluation dataset, retrieved contexts, generated answers, metric results, and run summary;
• asset checks for thresholds such as faithfulness, answer relevancy, context recall, and citation coverage;
• a small checked-in dataset so the example runs locally without external infrastructure;
• a clear separation between retrieval metrics and generation metrics, because they fail for different reasons;
• a regression comparison against the previous materialized evaluation result;
• tags/metadata for model name, embedding model, index version, prompt version, and corpus hash;
• a schedule or sensor showing how the same evaluation can run after index refreshes.

That would make it different from a normal RAG demo. The important lesson is not simply "run RAGAS"; it is that RAG quality should become a repeatable pipeline artifact with lineage, thresholds, and release decisions.

[bandish1304]

Hi @musaabhasan , This is extremely helpful feedback, I really like the idea of framing the example around orchestration, asset checks, and regression tracking rather than just wrapping a RAGAS script in Dagster. I’ll keep the scope focused on making RAG evaluation a repeatable Dagster-native pipeline artifact with metadata, thresholds, and local reproducibility.