Skip to content

v0.1.4

Latest
Latest

Choose a tag to compare

View all tags
@LakshyAAAgrawal LakshyAAAgrawal released this 15 Jul 14:50
Immutable release. Only release title and notes can be modified.
v0.1.4
8b0ce6c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This release ships the reflection-LM protocol and batch-based parallel proposals ([RFC #329](#329), Phases 1 & 4) — the largest change to GEPA's proposal machinery since the project started, contributed by @Benzhang2004 in #369 and hardened through five review rounds. One breaking change is called out under Breaking & upgrade notes.

pip install -U gepa

Highlights

  • Multi-proposal iterations via composable strategies (#369): each iteration can now sample and evaluate many candidate proposals instead of one. sampling_strategy= chooses the (parent, minibatch) tasks — SingleMutationSampling (default, classic GEPA), SameParentSampling(n), IndependentSampling(n), PxNSampling(p, n) — and selection_strategy= decides which improvements enter the pool: AllImprovements (default), BestImprovement, TopKImprovements(k). Defaults reproduce previous behavior exactly (pinned by replay tests against a recorded pre-change run). Full guide: [Parallel Proposals](https://gepa-ai.github.io/gepa/guides/parallel-proposals/).
  • The ReflectionLM protocol + reflection_strategy= (#369, RFC #329 Phase 1): the reflection step is now a seam. Pass a custom ReflectionLM implementation to gepa.optimize(reflection_strategy=...) or ReflectionConfig(reflection_strategy=...) to own how reflective mutation calls the reflection model — the entry point for session-based, aggregating (ComBEE-style, #307), and coding-agent reflectors. Stateful implementations return a successor from reflect(); chaining semantics are documented in [RFC #329](#329 (comment)). Multi-call reflectors get a ReflectionProposal.metadata diagnostics channel that flows through to callbacks, trackers, and proposal metadata.
  • batch_evaluator= — one external call for all pending evaluations (optimize_anything): hand GEPA a function that receives every pending (candidate, example) pair in a single call (provider batch APIs, Slurm/Ray, your own cluster) and return one result per pair. Parity with the standard evaluator path: shared evaluation caching (only misses reach your function, still as one call), per-pair opt_states warm-start injection, raise_on_exception policy with a documented _gepa_transient_failure contract (failures are never cached), str-candidate unwrapping, and identical output packaging. evaluator= is now optional when batch_evaluator= is provided — everything routes through your batch function, including refiner steps as singleton batches.
  • Adapters can batch: the optional GEPAAdapter.batch_evaluate seam lets adapters parallelize multi-proposal evaluations however they like; adapters that only implement evaluate() keep working unchanged (sequential fallback).
  • Selection is the authority, and rejection reasons are truthful: selection strategies receive all evaluated proposals plus the acceptance criterion (built-ins filter; custom strategies may deliberately surface criterion-rejected proposals). Everything unselected fires on_candidate_rejected with an accurate reason — acceptance failure, dropped-by-selection, or duplicate — and acceptance criteria may provide a reject_reason(proposal, state) hook for custom messages. Criteria are judged exactly once per proposal (memoized), so stateful or stochastic criteria behave correctly. Identical child candidates within an iteration are deduplicated before valset evaluation.
  • Per-task run traces: full_program_trace entries now record every task in multi-task iterations (n_tasks, tasks with per-task ids and before/after scores) and every accepted program index (new_program_indices); each proposal carries a stable metadata["proposal_id"]. Legacy keys are preserved for existing tooling.
  • Finer budget events: on_budget_updated now fires per evaluation stage (parent minibatch, child minibatch, valset) with self-consistent deltas — totals unchanged. New tracker metrics subsample/before and subsample/after join the existing subsample_score/new_subsample_score keys.

Fixed

  • optimize_anything merge proposals now fire GEPACallback events — the known issue from the v0.1.2 notes (#290 gap).
  • TrackingLM no longer hides batch_complete from wrapped callables that provide it, so batched reflection is not silently downgraded.
  • CI: matrix jobs no longer fail-fast, so a single-interpreter dependency failure can't masquerade as a five-version test failure.

Breaking & upgrade notes

  • EngineConfig.num_parallel_proposals is removed. The v0.1.2 notes anticipated a one-release deprecation shim; we're removing the parameter outright instead — it was experimental, shipped in only two patch releases, and the replacement is strictly more expressive. Migration: num_parallel_proposals=n → EngineConfig(sampling_strategy=SameParentSampling(n)) (or PxNSampling(p, n) to vary parents too), optionally with selection_strategy=.
  • Silent no-ops are now errors: reflection_strategy combined with adapter.propose_new_texts or custom_candidate_proposer raises ValueError (previously such conflicting configuration would be silently ignored); max_reflection_cost with a reflection_strategy that doesn't expose total_cost raises instead of silently never stopping.
  • Budget event granularity: if you aggregate on_budget_updated events, sum metric_calls_delta rather than counting events (counts per iteration increased; totals and monotonicity are unchanged).
  • Budget overshoot scales with proposal count: stop conditions are checked between iterations, so e.g. PxNSampling(4, 4) can overrun max_metric_calls by up to one full iteration's evaluations. See the guide's budget section.
  • Run directories from v0.1.3 load and resume normally; v0.1.4 adds only new trace keys.

Docs

Thanks

@Benzhang2004 (Jialin Zhang) authored #369 and carried it through five rounds of review — and makes his first contribution to GEPA with the largest feature PR in the project's history. Thanks also to @Shangyint for design review on RFC #329, and to everyone whose issues shaped the strategy interfaces (#61, #116, #298, #324).

What's Changed

New Contributors

Full Changelog: v0.1.3...v0.1.4

Contributors

LakshyAAAgrawal, Shangyint, and Benzhang2004
Assets 3
Loading