Skip to content

Conversation

@anistark
Copy link
Member

Changes Made

  • new cli command for quickstart templates
  • supported templates: rag_eval, agent_evals, benchmark_llm, prompt_evals and workflow_eval
Template Name Description
rag_eval RAG Evaluation Evaluate a RAG system with custom metric
agent_evals Agent Evaluation Evaluate AI agents with structured metrics and workflows
benchmark_llm LLM Benchmarking Benchmark and compare different LLMs with datasets
prompt_evals Prompt Evaluation Evaluate and compare different prompt variations
workflow_eval Workflow Evaluation Evaluate complex LLM workflows and pipeline

Testing

How to Test

  • Automated tests added/updated
  • Manual testing steps:
    1. uv run ragas quickstart
    2. uv run ragas quickstart rag_eval
    3. uv run ragas quickstart rag_eval --output-dir ./my-project

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 17, 2025
Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall this is great!
just a few questions

@anistark anistark merged commit dd1b2ac into explodinggradients:main Oct 20, 2025
8 checks passed
@anistark anistark deleted the feat/cli-templates branch October 20, 2025 04:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants