Keyhole

A multi-stage LLM query pipeline for generating high-quality answers using Google's Gemini AI.

Keyhole improves answer quality by expanding your query into multiple variations, generating parallel answers, evaluating and scoring each one, and selecting the best result through tournament-style comparison. Optional "overdrive" cycles iteratively refine the winning answer.

Features

• Query Expansion: Rewrites your query into multiple improved variations using prompt engineering techniques
• Parallel Generation: Generates answers for each variation concurrently
• Automated Evaluation: Scores each answer on a 0-10 scale for relevance, accuracy, and completeness
• Tie-Breaker Tournaments: ELO-style head-to-head comparisons when multiple answers tie for top score
• Overdrive Refinement: Optional iterative improvement cycles that critique and enhance the best answer
• Multiple Modes: Standard, Creative, Code, and Comparison modes for different use cases

Installation

1. Clone the repository:

   git clone https://github.com/yourusername/Keyhole.git
   cd Keyhole

2. Create a virtual environment (recommended):

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install dependencies:

   pip install -r requirements.txt

4. Set up your environment variables:

   cp .env.example .env

   Edit .env and add your Gemini API key:

   GEMINI_API_KEY=your_api_key_here
   

Usage

Run the pipeline interactively:

python keyhole.py

You'll be prompted for:

1. Query: Your question or task for the LLM
2. Overdrive cycles:
   • 0 - No refinement (fastest)
   • 1-3 - Number of iterative refinement cycles
   • c - Comparison mode (runs all 3 levels and saves results)
3. Creative mode (y/n): Allows answers to drift from the original intent for more creative results
4. Code mode (y/n): Uses specialized prompts optimized for code generation tasks

Example Session

  ────────────────────────────────────────
       K E Y H O L E   P I P E L I N E
       Multi-stage LLM Query Processor
  ────────────────────────────────────────

  * Gemini client initialized

> Enter your query:
  > Write a Python function to calculate fibonacci numbers

> Overdrive cycles (0=none, 1-3=cycles, c=compare):
  > 2

> Creative mode? (y/n, default: n):
  > n

> Code mode? (y/n, default: n):
  > y

Pipeline Stages

Stage 1: Query Expansion

Your query is rewritten into 7 improved variations using prompt engineering techniques including structured formatting, role definitions, and precise constraints.

Stage 2: Answer Generation

Each query variation is sent to Gemini in parallel, generating 7 candidate answers.

Stage 3: Evaluation

Each answer is scored from 0-10 based on how well it addresses the query requirements.

Stage 4: Tie-Breaker (Conditional)

If multiple answers share the top score, a round-robin tournament compares each pair head-to-head. The answer with the most wins is selected.

Stage 5: Overdrive (Optional)

The best answer is critiqued and rewritten multiple times. Each cycle aims to make the content more accurate, complete, and insightful.

Modes

Mode	Description
Standard	Evaluates all answers against your original query
Creative	Evaluates answers against their improved queries (allows creative interpretation)
Code	Uses code-specific prompts and saves results at each stage to files
Comparison	Runs 3 overdrive cycles, saving results at each level for analysis

Configuration

Configuration options are defined in keyhole.py:

CONFIG = {
    "NUM_PARALLEL_CALLS": 7,        # Number of parallel variations per stage
    "DEBUG": True,                   # Enable verbose output
    
    "MODELS": {
        "IMPROVEMENT": "gemini-3-flash-preview",
        "ANSWER_GENERATION": "gemini-3-flash-preview",
        "EVALUATION": "gemini-3-flash-preview",
        "TIE_BREAKER": "gemini-3-flash-preview",
    },
    
    "THINKING_LEVELS": {
        "IMPROVEMENT": "HIGH",
        "ANSWER": "HIGH",
        "EVALUATION": "HIGH",
        "TIE_BREAKER": "HIGH",
    },
}

Environment Variables

Variable	Required	Description
GEMINI_API_KEY	Yes	Your Google Gemini API key
KEYHOLE_DEBUG	No	Set to 1, true, or yes to enable debug output

Output

Standard/Creative Mode

Returns the best answer and its score directly to the terminal.

Code Mode

Creates an output folder with:

• no_overdrive{.ext} - Initial best answer
• overdrive_1{.ext} - First refinement
• overdrive_2{.ext} - Second refinement (if applicable)
• *_meta.txt - Metadata for each result
• code_summary.txt - Summary of the run

Comparison Mode

Creates an output folder with results from each overdrive level for side-by-side analysis, plus a comparison_summary.txt with scores.

Requirements

• Python 3.8+
• Google Gemini API key
• Dependencies listed in requirements.txt:
  • google-genai>=1.59.0
  • python-dotenv>=1.2.1

License

MIT License