v0.1.0
This is the initial alpha release of PromptCache, a lightweight middleware designed to reduce LLM costs and latency by intelligently caching responses based on semantic intent.
✨ Key Features
🧠 Smart Semantic Matching: Uses a two-stage verification strategy (similarity thresholds + intent check) to ensure cached responses are accurate.
🔌 OpenAI Compatibility: Works as a drop-in replacement for the OpenAI API—just change your base_url.
⚡ High Performance: Written in pure Go with BadgerDB for fast, embedded persistent storage.
🐳 Docker Ready: Includes
Dockerfile
and
docker-compose.yml
for instant deployment.
This release is in Alpha. While functional, it is not yet production-ready. We are actively working on stability, configuration, and performance improvements for v0.2.0.