Skip to content

Releases

v0.2.0

Choose a tag to compare

@agunapal agunapal released this 21 Apr 15:00
a06dceb

What's new in v0.2.0

Pipeline mode

Optimize prompts that live inside multi-step agent pipelines — classify → retrieve → generate — not just single LLM calls. Standard mode scores a string against an ideal; pipeline mode re-runs your full agent on every candidate prompt and scores the complete execution trace.

How it works: wrap your existing agent in a pipeline_fn(prompt, input) -> AgentTrace function — your pipeline code doesn't change. Mark the node being optimized with optimize=True; everything else is passed to the judge as grounding context.

  • AgentTrace and TraceNode types for structured trace recording
  • set_pipeline() + set_inputs() Python API
  • --pipeline-file + --inputs-file CLI flags
  • AgentTrace.tokens field for tracking pipeline token usage alongside judge tokens
  • Per-sample timeout with graceful 0.0 scoring on context-window overflow
  • All four strategies work in pipeline mode, including PDO

CLI

aevyra-reflex optimize prompt.md \
  --pipeline-file pipeline.py \
  --inputs-file   inputs.json \
  --reasoning-model openrouter/qwen/qwen3-8b \
  --judge           openrouter/qwen/qwen3-30b-a3b \
  --judge-criteria  criteria.md

Language detection

Reflex now detects the dominant language of the training set at run start using Unicode character-range heuristics. The reasoning model writes prompt revisions and diagnostic explanations in the same language as the dataset — preventing multilingual models from silently switching languages mid-optimization. A 15% threshold guards against mislabelling English text that contains quoted foreign terms.

Dashboard fix

Fixed an unterminated string literal in the empty-state UI that caused the dashboard to show a blank loading screen instead of the run list.

Docs

  • New Modes guide — when to use standard mode vs pipeline mode
  • New Pipeline mode tutorial — optimizing a tool-calling developer assistant end-to-end with real run logs

Install

pip install aevyra-reflex==0.2.0

v0.1.0

Choose a tag to compare

@agunapal agunapal released this 14 Apr 15:56
f2a616c

aevyra-reflex is an agentic prompt optimizer that iterates on your prompt until it hits the target score.

What's included in v0.1.0

Optimization engine

  • Iterative prompt optimization with train/val/test split and early stopping on val plateau
  • Mini-batch sampling for faster iterations
  • Statistical significance testing across multi-run evals
  • Resume interrupted runs from last checkpoint

Strategies

  • Iterative rewriting with diagnosis of eval failures
  • Structural transforms (reorder, rephrase, expand, compress) with transform history
  • Adaptive multi-ranker fusion with learned method weights (PDO)
  • Label-free aware diagnosis and refinement

Providers

  • OpenAI, Anthropic, Google Gemini, OpenRouter
  • Local models via any OpenAI-compatible API (Ollama, vLLM)
  • --source-model flag for model migration context

Integrations

  • MLflow: per-iteration score tracking (train/val/test), best prompt logging
  • W&B: per-iteration charts with score_train, score_val, score_test series

CLI

  • aevyra-reflex optimize — run prompt optimization
  • --mlflow / --wandb flags for experiment tracking
  • --judge-criteria flag for custom eval criteria
  • Dashboard for run history and analysis

Docs

Install

pip install aevyra-reflex