The prompt engineering, prompt management, and prompt evaluation tool for TypeScript, JavaScript, and NodeJS.
-
Updated
Jun 19, 2024 - TypeScript
The prompt engineering, prompt management, and prompt evaluation tool for TypeScript, JavaScript, and NodeJS.
Test your prompts, agents, and RAGs. Use LLM evals to improve your app's quality and catch problems. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.
Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
Official implementation for the paper *🎯DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving*
Open-Source Evaluation for GenAI Application Pipelines
Superpipe - optimized LLM pipelines for structured data
The LLM Evaluation Framework
The all-in-one LLM developer platform: prompt management, evaluation, human feedback, and deployment all in one place.
Python SDK for running evaluations on LLM generated responses
A framework to build scenario simulation projects where human and LLM based agents can participant in, with a user-friendly web UI to visualize simulation, support automatically evaluation on agent action level.
The prompt engineering, prompt management, and prompt evaluation tool for Python
TypeScript SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)
Awesome papers involving LLMs in Social Science.
The prompt engineering, prompt management, and prompt evaluation tool for C# and .NET
The prompt engineering, prompt management, and prompt evaluation tool for Ruby.
The prompt engineering, prompt management, and prompt evaluation tool for Kotlin.
The prompt engineering, prompt management, and prompt evaluation tool for Java.
Add a description, image, and links to the llm-evaluation topic page so that developers can more easily learn about it.
To associate your repository with the llm-evaluation topic, visit your repo's landing page and select "manage topics."