v0.5.0

Replaces the evaluation stack with an Inspect AI package, adds .spipe as a portable serialization format for SteeringPipeline, and renames the package from aisteer360 to steerability. Additionally ports the toolkit to transformers v5 (Python 3.12+) and broadens the model families that state controls and activation capture run on.

Evaluation

The Benchmark, metrics, and use-case modules are replaced by an Inspect AI evaluation package (steerability/evaluation/, behind the new eval extra). Target-behavior evaluations are ordinary Inspect Tasks; the toolkit ships no task, scorer, or metric classes of its own.

• as_inspect_model(pipeline) and ProviderOptions: a generation-only Inspect model provider over a steered pipeline. Every generation routes through SteeringPipeline.generate(); prompts enter as messages= when the tokenizer has a chat template (so adapt_messages controls fire) and as rendered text= otherwise, with the path recorded as prompt_path in provenance. Logprob parameters, tools, and multimodal content are refused with actionable messages
• Concurrent Inspect requests collate into batched pipeline calls when every enabled control is batch-safe (evaluation/batching.py); a seeded dispatch carries seed_scope from ProviderOptions (default "dispatch"), so a seeded batch decodes in one pass on the Hugging Face backend
• runtime_kwargs_solver delivers per-sample runtime kwargs from Sample.metadata; sample_scorer_from_inspect adapts any Inspect scorer into a per-row SampleScorer reward; InspectSuite names a task set over eval_set (limit, sample_ids, task_args, generate_overrides, retry_attempts)
• SteeringEval runs configurations (fixed controls, ControlSpec sweeps, and the empty-list baseline arm) x trials x suites, one GPU-resident pipeline at a time, with a pre-flight check() over every sweep point (on_unsupported="raise" | "skip") and one derived seed per (configuration, trial). There is no separate results checkpoint: the .eval logs under save_dir/inspect_logs/ are the store, and eval_set resumes each cell at sample granularity. Results reshape through results(), runs_frame(), samples_frame(), and summarize_runs
• viz_utils is reworked into evaluation/plotting.py over the summarized runs frame (plot_metric_by_config, plot_tradeoff_scatter, plot_metric_heatmap, plot_comparison_bars, plot_sensitivity, plot_tradeoff, plot_pareto_frontier)
• The sweep layer has no Inspect dependency: identity.py moves into algorithms/core/, alongside the new core/sweeps.py (expand_configurations, preflight, PipelineFactory) and core/scoring.py (SampleScorer)
• Every RUNTIME_KWARGS_SCHEMA entry declares a scope ("row" for a per-prompt value delivered row-aligned in batched calls, "call" for one value per call), validated at steer() and read by the provider; SearchDriver reward_params is row-scoped (one mapping per row, merged into the scorer's params)
• The benchmark notebooks are ported to Inspect as examples/notebooks/studies/ with task modules (commonsense_mcqa, instruction_following), and a routing-vs-prompting study is added

Pipeline serialization

A new steerability/spipe/ package adds .spipe, a portable serialization format for SteeringPipeline (spipe/1 manifest; a zip when the path ends in .spipe, a directory otherwise).

• A spipe stores the recipe (model reference plus controls as constructed) and, once the pipeline is steered, the frozen resolution (fitted vectors, probes, adapters, optimized prompts) in a content-addressed artifact store with a lock section (fingerprints, per-fit digests)
• SteeringPipeline.to_spipe() builds one; SPipe.load, save (artifacts="thin" writes the manifest only, resolved at load through artifact_store=), pipeline, verify, and thaw round-trip it. Frozen entries instantiate from their resolution, so steer() on a loaded pipeline is cheap and model-free; prefer="recipe" forces re-fits, allow_code=True gates callable references, non-toolkit dataclasses, and pickle-backed memories, and a stale bundle (fit-relevant recipe fields edited after freezing) raises unless allow_stale=True
• Controls declare how they freeze through steer_fits(), export_state(), frozen_form(), and fit_identity(); InterventionControl derives all four from its template, and a control that produces state without a frozen form raises NotFreezableError
• Trained structural controls freeze as the new load_checkpoint / load_lora controls; intervention controls freeze as activation_adapter entries unless they declare a same-class frozen form (caa, act_add, iti)

New methods

• system_prompt: sets or merges the leading system message of a chat (prepend, append, or replace), always producing exactly one system message
• user_prefix: prepends a fixed text marker to a user turn (the last, first, or all user turns), with the token stream as a fallback for non-chat input
• load_checkpoint and load_lora: structural controls that load a checkpoint directory or a LoRA adapter (merge=False keeps the PEFT wrapper, so a state control listed after it steers the adapted model); these are the frozen forms of trained structural controls in a .spipe

Other changes

• Dependencies: ported to transformers v5 with the ML stack bumped (trl 1.0, peft 0.20, datasets 4.7, accelerate 1.4, vllm 0.26 with trl[vllm], mergekit from a pinned commit); Python 3.12+ required; uv.lock is committed
• Model families: new core/internals/model_layout.py (resolve_model_layout, text_config, register_layout_detector, per-layer head_geometry) resolves the decoder stack at model.layers, model.language_model.layers, or transformer.h; state controls and activation capture support composite multimodal wrappers (Gemma 3), unmerged PEFT/LoRA wrappers, and hybrid attention stacks (Qwen3.5 / Qwen3-Next, where residual-stream controls work unchanged and head-level controls refuse the non-attention layers)
• Generation: batched prompts are left-packed on the HF session and before state-hook assembly in _execute_generation, so full-sequence returns of a padded batch are left-packed ([pads, prompt, continuation]); seed_scope ("item" or "dispatch") maps a generate(seed=...) onto the items of a multi-item dispatch; Output.generated_tokens and DecodingDriver.max_rollouts_per_query() count the tokens and continuations a driver produces across rollouts
• Decoding drivers: the phased, routed, and budget-forcing paths are fixed across the backend interface, and finished-beam detection in the search Frontier is fixed
• PASTA: head profiling as a steer-time fit (HeadProfile, resolved at steer() to a head map); scale_position and the decode-phase attention mask are fixed under transformers v5
• RAD: prefix-reward training (rad/utils/reward_training.py, with the loss computed in float32) and sequence-classification heads for the Granite families; the reward head is read at the candidate position, so an eos candidate no longer receives the prefix's reward
• SASA fits from contrastive pairs (gen_wv_data accepts LabeledExamples or ContrastivePairs); the ITI estimator is reworked
• vLLM: new backends/vllm/environment.py turns the FlashInfer sampler off by default at offline engine boot (an explicit caller value wins), forces VLLM_HOOK_WORKER=unified for hook_plugin engines, and provides serve_environment for launched servers; vllm-hook-plugins installs from the [steerability-interface](https://github.com/emiehling/vLLM-Hook/tree/steerability-interface) branch
• TRL wrappers: prompt_format on the DPO/APO preference path (with a boundary warning), training_args validated against the installed TRL config (an undeclared key raises at construction), DPO multi-loss (loss_type as a list with loss_weights), and resume_from_checkpoint passed to trainer.train()
• Reasoning models: split_thinking_ids splits on token ids when the delimiters are special tokens (ProviderOptions.reasoning_split="auto" | "text" | "tokens", with reasoning_opened_at_start for templates that open the channel in the generation prompt); utils/answers.py adds numeric answer extraction
• canonical_value recurses into the elements of NumPy object arrays, so config_digest no longer raises on object arrays arriving from a pandas or datasets row
• Docs: the evaluation tutorial is rewritten (evaluate_steering_pipelines.md); the spipe concept page and reference pages for evaluation, backends, spipe, utils, and the new controls are added; a bibtex-warning hook (docs/_hooks/) and a docs-nav test are added, mkdocs build --strict passes, and AGENTS.md and README are rewritten
• Notebooks: the generics and library-wrapper notebooks move under algorithms/ (generics/, wrappers/); system_prompt, honest-persona prompting, routing-vs-prompting, vLLM serve, and working-with-spipes notebooks are added; the SASA (style self-steering), RAD (prefix-reward detoxification on Granite 4.1), best-of-n, and budget-forcing notebooks are refreshed; all notebooks rerun
• Tests: added for the Inspect provider, batching, solvers, scorers, suite, runner, and frames; spipe codec, format, store, identity, and freeze; sweeps and the runtime-kwargs schema; Gemma 3, hybrid-attention, and LoRA-composition steering; the new controls; the DPO wrapper, TRL resume, and training-args validation; ITI, SASA, and RAD reward training; the search frontier, reward params, phased token boundaries, max rollouts, generated tokens, and padding alignment; the vLLM engine environment; the docs nav

Breaking changes

• Package rename: aisteer360 becomes steerability for the package directory, distribution, identifiers, and documentation (e.g. aisteer360.algorithms.core.steering_pipeline → steerability.algorithms.core.steering_pipeline); the spipe lock records the toolkit under the new name
• Evaluation: Benchmark, the metrics package (LLMJudgeMetric, Perplexity, ...), the UseCase subclasses, viz_utils, and the generate_on_pipeline() / batch_retry_generate() helpers are removed in favour of steerability/evaluation/. Benchmark checkpoints and checkpoint_every are gone (the .eval logs are the store), so 0.4.0 checkpoints are not resumed. The TruthfulQA composite study is removed
• Python 3.12+ is required, and the dependency ranges move to transformers 5.x, trl 1.x, peft 0.20, datasets 4.7+, accelerate 1.4+, and vllm 0.26+
• Packaging: the plots, guided, and cpo extras are removed (xgrammar is a core dependency; econml for CPO's use_dml is installed separately); eval (Inspect AI, inspect-evals, matplotlib, seaborn) is the new workflow extra and all installs it; merging is declared a uv conflict with eval, all, and vllm; contributor tooling moves from extras into [dependency-groups] (dev, notebooks, docs), so dev is no longer an extra and uv sync installs it by default
• SearchDriver presets (DeAL, BestOfN, SearchDecoding) take reward_params per row: a batched call passes one mapping per row rather than one mapping for the call
• identity.py moves to steerability/algorithms/core/identity.py

What's Changed

• Refactor evaluation to use Inspect, bump package versions, rename package to steerability by [@emiehling](https://github.com/emiehling) in [#25](#25)

Full Changelog: [v0.4.0...v0.5.0](v0.4.0...v0.5.0)