AgentBait

You Won't Believe This Click
Content Rewriting for Agentic Choice

Tianyi Jin · Zirui Wang · David M. Chan
University of California, Berkeley

Project page & paper · Quick start · Reproducibility · Citation

---

Same slate. One rewrite. Can the decision change?

AgentBait studies how changing only one item's presentation—while keeping the candidate slate fixed—can change an LLM agent's choice. A learned advisor proposes a rewriting strategy, a frozen rewriter edits the target item's title and abstract, and the target agent chooses again from the otherwise unchanged slate.

ORIGINAL 17.1% target selected

→

PROMPT REWRITER 34.8% target selected

→

TRAINED ADVISOR 98.5% target selected

_{Paper-reported results on the main English MIND evaluation.}

At a glance

Fixed slate	Advisor → rewriter	Selection + support
Exactly one target item is rewritten; competing items stay fixed.	The advisor proposes a strategy and the frozen rewriter realizes it.	We measure selection shift and can add source-support rewards.

The central finding is double-edged: agent feedback creates a learnable pressure on presentation, but optimizing selection alone can discover unsupported shortcuts. AgentBait therefore treats source support as a first- class experimental variable rather than assuming that selection implies usefulness or truthfulness.

Quick start

The supported interpreter is Python 3.12 or newer. Install uv, clone the repository, and run the offline checks:

git clone https://github.com/Agentbait/agentbait.git
cd agentbait
uv sync --locked --dev
make smoke
make training-smoke-cpu

These commands use synthetic data and require neither a GPU nor an API key. They import the package, construct a real two-row environment batch, execute environment steps, and compute the final-selection reward with a deterministic offline evaluator.

For the complete CPU CI path:

uv sync --locked --dev --extra figures --extra local-eval
make check
make build

The CPU smoke deliberately does not load policy weights, initialize CUDA or SkyRL workers, run model rollouts, or perform an optimizer step.

How AgentBait works

1. Fix the decision context. Select one target from an already-constructed candidate slate.
2. Advise. The trainable advisor proposes a global rewriting strategy after seeing the target and its competitors.
3. Rewrite one item. A frozen rewriter updates only the target title and abstract.
4. Choose again. A target agent selects from the updated slate.
5. Learn from the choice. Selection reward trains the advisor; an optional source-support term changes what strategies are rewarded.

Paper-to-code map

Paper concept	Maintained implementation
Fixed candidate slate and deterministic labels	advisor_models/rewrite/config.py
Advisor-guided environment and reward	advisor_models/rewrite/envs/advisor.py
Direct standalone rewriter baseline	advisor_models/rewrite/envs/standalone.py
Advisor, rewriter, and chooser prompts	advisor_models/rewrite/prompts.py
Source-support scoring and fail-fast behavior	advisor_models/rewrite/nli_utils.py
SkyRL trainer and generator integration	advisor_models/rewrite/trainer.py, advisor_models/rewrite/skyrl_gym_generator.py
API/local evaluation and aggregation	advisor_models/rewrite/eval_rewrite.py, scripts/eval_rewrite_local.py
MIND-style preprocessing	scripts/datasets/prepare_mind_impressions.py, scripts/datasets/build_mindlarge_final_selection.py

Large datasets, checkpoints, model exports, API caches, W&B runs, and raw training outputs are intentionally excluded from Git.

Data, training, and evaluation

Generate a safe synthetic fixture

uv run --locked python scripts/datasets/generate_synthetic_smoke_data.py \
  --output-dir /tmp/agentbait-smoke-data

Build a rewrite dataset

Use only source data whose terms permit local processing:

uv run --locked python -m advisor_models.rewrite.datasets.advisor \
  --input_path /path/to/input.jsonl \
  --output_dir /path/to/rewrite-data

The standalone builder is available at advisor_models/rewrite/datasets/standalone.py.

Evaluate with hosted models

uv run --locked python -m advisor_models.rewrite.eval_rewrite \
  --dataset_path /path/to/constructed.jsonl \
  --advisor_model gpt-5-mini \
  --rewriter_model gpt-5-mini \
  --evaluator_model gpt-5-mini

This path requires the provider credential expected by the selected backend. Hosted model outputs are not bitwise reproducible because provider weights and serving stacks are not content-addressed.

Evaluate local exports

uv sync --locked --extra local-eval
uv run --locked python scripts/eval_rewrite_local.py \
  --dataset_path /path/to/constructed.jsonl \
  --variant standalone \
  --rewriter_model /path/to/policy_merged \
  --evaluator_model /path/to/evaluator

GPU training and checkpoint export

GPU training uses the separately pinned SkyRL environment documented in docs/setup_skyrl.md. The base checkout intentionally does not vendor SkyRL or CUDA packages.

Advisor training entrypoint:

export DATA_DIR=/path/to/constructed-dataset
export ADVISOR_RUNS_ROOT=/path/to/persistent/run-storage
bash scripts/train_rewrite_advisor.sh

Standalone training entrypoint:

export DATA_DIR=/path/to/constructed-dataset
export ADVISOR_RUNS_ROOT=/path/to/persistent/run-storage
bash scripts/train_rewrite_standalone.sh

Checkpoint export:

bash scripts/export_ckpt_to_hf.sh \
  --checkpoint-dir /path/to/ckpts/run/global_step_100 \
  --export-root /path/to/exports/run

These commands describe the maintained training interface. They do not by themselves reproduce the exact historical paper run.

Reproducibility and artifacts

Start with REPRODUCIBILITY.md, which separates:

• checks that run from a clean clone without a GPU;
• the maintained SkyRL compatibility contract;
• licensed or externally hosted data and model dependencies;
• legacy audit metadata retained from an earlier experiment snapshot; and
• artifacts still required before claiming full paper reproduction.

Machine-readable records live under artifacts/. The current 4B aggregate and its historical paper_* identifiers are retained only as a legacy audit snapshot for compatibility. They do not correspond to the current manuscript's Qwen3.5-9B results, sample counts, figures, or tables.

NLI/source-support scorer errors fail fast by default so training cannot silently change its reward objective. An explicit fallback must be enabled by REWRITE_NLI_ALLOW_BASE_REWARD_FALLBACK_ON_ERROR=true, and fallback use is recorded in trajectory metadata.

Repository map

advisor_models/rewrite/   maintained environments, prompts, trainer, evaluation
configs/experiments/      small set of smoke, template, and legacy audit configs
scripts/datasets/         data preparation and synthetic smoke generation
scripts/experiments/      advisor and standalone launchers
scripts/                  evaluation, export, manifest, and smoke utilities
tests/                    CPU unit, contract, and release checks
artifacts/                manifests plus explicitly labeled legacy audit metadata
docs/                     SkyRL setup and README visual assets

Version 0.1.0 intentionally excludes the older math, reviews, RuleArena, SWE-Smith, and vision task families. The maintained surface is text-only rewriting.

Limitations and responsible use

• AgentBait studies text-only, fixed-slate choice. It is not a retrieval, ranking, recommendation, or production deployment system.
• Optimizing target selection can encourage manipulative or unsupported presentation. The code is released to study and audit that pressure, not to endorse deceptive content optimization.
• A source-support reward is an experimental proxy, not a guarantee of factuality, usefulness, safety, or human benefit.
• Hosted-agent behavior can change over time. Record model identifiers, request dates, decoding settings, prompts, and raw outputs for every run.
• Respect the licenses, privacy terms, and redistribution restrictions of all upstream datasets and model checkpoints.

Citation

If this code or project is useful in your research, please cite the paper:

@article{jin2026agentbait,
  title  = {You Won't Believe This Click: Content Rewriting for Agentic Choice},
  author = {Jin, Tianyi and Wang, Zirui and Chan, David M.},
  year   = {2026},
  url    = {https://agentbait.github.io/}
}

Machine-readable citation metadata is available in CITATION.cff.

Acknowledgements and license

The maintained trainer and generator integration extends NovaSky-AI/SkyRL at the revision recorded in NOTICE. SkyRL is installed separately and is not vendored into this repository.

AgentBait is released under the Apache License 2.0. Contributions are welcome; see CONTRIBUTING.md.