In the rapidly evolving field of AI and large language models, crafting effective prompts has become a crucial skill. Blogus is born out of the need for a sophisticated, user-friendly tool that empowers developers, researchers, and AI enthusiasts to:
- Analyze and refine prompts in real-time
- Generate and manage test cases
- Execute prompts across multiple AI models
- Visualize and understand prompt effectiveness
- Separate target models (for execution) from judge models (for analysis)
- Manage the complete prompt development lifecycle
Whether you're a seasoned prompt engineer or just starting your journey with AI, Blogus provides the tools you need to elevate your prompt crafting skills.
Blogus is an advanced prompt engineering tool available as:
- A Python library for programmatic use in your applications
- A command-line interface for quick analysis and testing
- A web interface with an intuitive frontend for interactive prompt engineering
Key features include:
- Real-time Prompt Analysis: Get instant feedback on your prompts' effectiveness and alignment with your goals.
- Multi-model Support: Test your prompts across various AI models, including GPT-4, Claude, and more.
- Test Case Generation: Automatically generate relevant test cases for your prompts.
- Interactive Code Editor: A feature-rich editor with syntax highlighting and real-time analysis feedback (in web UI).
- Execution Environment: Run your prompts and see the results immediately.
- Test Case Management: Store and download generated test cases as JSON files (in web UI).
- Dynamic Goal Inference: Automatically infer the goal of your prompt if not provided.
- Target vs Judge Models: Separate models for prompt execution and analysis for optimal cost and performance.
- Complete Prompt Lifecycle: Tools for the entire prompt engineering workflow from creation to optimization.
Comprehensive documentation is available in the docs directory:
- Getting Started - Installation and basic setup
- Core Concepts - Understanding the fundamentals of Blogus
- Prompt Development Lifecycle - Complete workflow for prompt engineering
- Command Line Interface - Using Blogus from the command line
- Web Interface - Using the web-based interface
- Library Usage - Integrating Blogus into your Python applications
- Model Selection - Choosing the right models for your tasks
- Advanced Features - Advanced techniques and workflows
- API Reference - Detailed API documentation
- Contributing - How to contribute to Blogus
Check out the examples directory for practical usage examples:
- Basic Analysis - Simple prompt analysis and execution
- Test Generation - Generating test cases for parameterized prompts
- Cross-Model Comparison - Comparing prompt responses across different models
pip install blogusFor web interface support, install with the web extra:
pip install blogus[web]from blogus.core import TargetLLMModel, JudgeLLMModel, analyze_prompt, generate_test
prompt = "You are an AI assistant that helps people find information..."
# Analyze a prompt using a judge model
analysis = analyze_prompt(prompt, JudgeLLMModel.GPT_4)
print(f"Goal alignment: {analysis.overall_goal_alignment}/10")
# Generate a test case using a judge model
test_case = generate_test(prompt, JudgeLLMModel.GPT_4)
# Execute a prompt using a target model
result = execute_prompt(prompt, TargetLLMModel.GPT_4)# Analyze a prompt with separate target and judge models
blogus analyze "You are an AI assistant that helps people find information..." \\
--target-model gpt-4o \\
--judge-model claude-3-opus-20240229
# Generate a test case with separate target and judge models
blogus test "You are an AI assistant that helps people find information..." \\
--target-model gpt-4o \\
--judge-model claude-3-opus-20240229
# Execute a prompt with a target model
blogus execute "You are an AI assistant that helps people find information..." \\
--target-model gpt-4o
# Infer the goal of a prompt with a judge model
blogus goal "You are an AI assistant that helps people find information..." \\
--judge-model claude-3-opus-20240229
# See all available commands
blogus --helpTo run the web interface:
blogus-web # If installed with web extras
# or
python -m blogus.webThen navigate to http://localhost:8000 in your browser.
Blogus uses LiteLLM to support a wide range of AI models. You'll need API keys for the models you want to use:
- OpenAI API key for GPT models
- Anthropic API key for Claude models
- Groq API key for Mixtral/Llama models
Set these as environment variables:
export OPENAI_API_KEY=your_openai_api_key
export ANTHROPIC_API_KEY=your_anthropic_api_key
export GROQ_API_KEY=your_groq_api_keyFor other models supported by LiteLLM, please refer to the LiteLLM documentation for the required environment variables.
We welcome contributions to Blogus! Please see our CONTRIBUTING.md for details on how to get started.
Blogus is open-source software licensed under the MIT license. See the LICENSE file for more details.
If you encounter any issues or have questions, please file an issue on our GitHub issue tracker.
If you use Blogus in your research, please cite it as follows:
@article{sarkar2024blogus,
title={Blogus: An Advanced Tool for Crafting, Analyzing, and Perfecting AI Prompts},
author={Sarkar, Dipankar},
journal={arXiv preprint arXiv:2024.xxxxx},
year={2024},
url={https://github.com/terraprompt/blogus},
note={Software available from https://github.com/terraprompt/blogus},
abstract={Blogus is an open-source software tool designed to facilitate the
process of prompt engineering for large language models. It provides
real-time analysis of prompts, supports multiple AI models, and offers
features for test case generation and prompt execution. This tool aims to
enhance the efficiency and effectiveness of prompt crafting in AI research
and applications.}
}For LaTeX users, you can use the following command to cite Blogus:
\cite{sarkar2024blogus}Happy Prompt Engineering with Blogus! 🚀