Quick Start

This is the fastest path from a fresh installation to a repeatable evaluation loop: configure providers, create a prompt, compare outputs, and turn the useful cases into a suite you can rerun.

flowchart LR
    A[Configure Providers] --> B[Create Prompt]
    B --> C[Choose Native or App Callback Execution]
    C --> D[Run Across Models]
    D --> E[Create or Reuse a Dataset]
    E --> F[Create Test Suite]
    F --> G[Track Evolution and Export Results]

Loading

1. Configure Providers

Start with one local model and one hosted model if you want a quick read on the balance between quality, cost, and speed.

Provider	Suggested starter model	Requirement
Ollama	llama3.1:8b or another local model	http://localhost:11434/api/chat available
OpenAI	gpt-4o	OPENAI_API_KEY
Anthropic	claude-sonnet-4-5-20250929	ANTHROPIC_API_KEY
Google Gemini	gemini-2.5-flash	GOOGLE_API_KEY

See Providers for complete configuration details.

2. Create a Prompt

Templates use {{variable}} placeholders so a single prompt can be reused across many inputs without losing its form:

Summarize the following article in 3-5 bullet points:

{{article}}

3. Run the Prompt

1. Select one or more providers.
2. Supply variable values for the template.
3. Choose whether Aludel should call the provider directly or route execution through your app callback.
4. Execute the run and compare outputs side by side.

Each result captures the marks left by the run:

• Response text
• Latency in milliseconds
• Input and output token counts
• Estimated cost in USD
• Execution status per provider result
• Optional callback metadata when your app returns it

In callback mode, token, latency, and cost fields may be absent. The run views still render cleanly and show N/A where your executor omits a metric.

4. Create a Test Suite

Optionally create a reusable dataset first. Dataset entries can carry variables, multi-turn messages, assertions, and metadata; populating a suite records provenance and skips entries already imported.

Turn an ad hoc prompt into something you can assay repeatedly:

{
  "name": "Phoenix LiveView Summary",
  "variable_values": {
    "article": "Phoenix LiveView is a library..."
  },
  "assertions": [
    {"type": "contains", "value": "real-time"},
    {"type": "regex", "value": "^- .*\\n- .*\\n- .*"}
  ]
}

Supported assertion types:

• contains for substring checks
• not_contains for absence checks
• regex for pattern validation
• exact_match for exact output comparisons
• json_field for typed structured response validation
• json_deep_compare for partial JSON scoring with an optional threshold

Test cases can also include uploaded documents such as PDFs, images, CSV, JSON, and plain text files. Aludel stores the document metadata in PostgreSQL and persists the file contents through the configured storage backend.

5. Track Evolution

Every prompt update creates a new version, which makes it possible to compare:

• Success rate across versions
• Latency trends over time
• Cost changes by provider and prompt version
• Average structured-output score across suite runs
• Provider-specific execution failures and partial failures

Use the Evolution view to see whether each change refined the prompt or introduced fresh impurities.

Run and suite result pages also include copy actions and raw JSON export endpoints, which makes it easier to hand off structured evidence or inspect assertion details outside the UI.

Next Steps

• Prompts for template syntax and versioning
• Providers for provider-specific options
• Datasets for reusable evaluation inputs
• Evaluation Suites for assertion and retry examples
• Exports and CI for automation
• Architecture for the execution model behind the UI