🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23

Test your prompts, models, and RAGs. Catch regressions and improve prompt quality. LLM evals for OpenAI, Azure, Anthropic, Gemini, Mistral, Llama, Bedrock, Ollama, and other local & private models with CI/CD integration.

An open-source visual programming environment for battle-testing prompts to LLMs.

The production toolkit for LLMs. Observability, prompt management and evaluations.

🔰Visual Studio Code Extension For Compiling Language

A command line tool for diffing json rest APIs

🤖 Build AI applications with confidence ✅ Understand how your users are using your LLM-app ✅ Get a full picture of the quality performance of your LLM-app ✅ Collaborate with your stakeholders in ONE platform ✅ Iterate towards the most valuable & reliable LLM-app.

Safely execute untrusted code with ESM syntax support, dynamic injection of ESM modules from URL or plain JS code, and granular access control based on whitelisting for each JS object.

A highly configurable custom expression tree evaluator

An application to help to make good career choices

Tool to evaluate how FAIR is a resource URL using the F-UJI API

Continual development workflow developed for HackFS 2023

Primary School Academic Information Management System, PSAIMS makes it easy to collect, process, analyse and disseminate Tanzanian based primary school academic information.

Programming Language Selector based on language metadata and user-specified values.

Execute JavaScript in Koishi

Integrate FigTree Evaluator with JSON Forms

Package for transforming a string with logical operators into the result of an expression

Code for the paper "Big City Bias: Evaluating the Impact of Metropolitan Size on Computational Job Market Abilities of Language Models" (NLP4HR '24)

League Of Legends Statistics Tool