PREF-VALID

When should you trust an LLM's preference?

A measurement framework that elicits one latent preference through four independent channels (direct rating, forced pairwise choice, resource allocation, revealed behavioral choice) across 50 value-conflict scenarios, then tests whether agreement between methods predicts what a model does under a method it was never calibrated against. The answer, across 3,600 API calls and two models, is yes: convergence behaves as a calibrated confidence signal (gpt-4o: r = 0.54, p < 0.001; gpt-4o-mini: r = 0.17, p = 0.016).

Full writeup: report.md · report/report.pdf (LaTeX-typeset)

The idea in one picture

Three of four elicitation methods calibrate a predicted preference; their agreement becomes a convergence score C. The fourth method, held out and rotated across all four positions, is what that prediction is checked against. Sort every scenario by C and trust the highest-convergence predictions first, and accuracy stays near 1.0 well past half coverage before degrading to the trust-everything baseline.

Key results

	gpt-4o-mini	gpt-4o
Unseen-method prediction accuracy	0.900	0.830
r(convergence, accuracy)	0.165 (p = 0.016)	0.541 (p < 0.001)
Selective-prediction AURC gain vs. random	+0.059	+0.141
Cross-model scenario agreement	0.84 (42/50 scenarios)

See report.md §5 for the full results, including robustness under framing/persona/ sampling/position perturbation, Cohen's κ, Expected Calibration Error, and baselines.

Repository layout

prefer/
├── README.md              this file
├── report.md               full paper (Markdown, GitHub-renders the math)
├── requirements.txt
├── config.py                model list, perturbation settings
├── client.py                OpenAI API wrapper with disk caching
├── elicitors.py              the four elicitation mechanisms
├── metrics.py                convergence, stability, robustness, ECE, AURC, leave-one-out
├── run.py                    orchestrates the full experiment (3,600 API calls)
├── analyze.py                computes metrics + generates figures/report per model
├── scenarios/
│   └── scenarios.json        50 value-conflict scenarios, tagged by tradeoff strength
├── results/
│   ├── raw.json               every API response (with model reasoning), tracked in git
│   ├── cross_model_comparison.json
│   ├── figures_combined/      cross-model comparison figure
│   ├── gpt-4o/                 metrics.json, report.txt, figures/ for gpt-4o
│   └── gpt-4o-mini/            same, for gpt-4o-mini
└── report/
    ├── report.tex              LaTeX source (compiles with pdflatex)
    ├── report.pdf               compiled paper
    └── figures/                 figures embedded in the paper

results/cache.json and .env are gitignored: the cache is a regenerable resumability artifact, and .env holds your API key.

Reproducing

pip install -r requirements.txt

# create prefer/.env with:
# OPENAI_API_KEY=sk-...

python run.py       # ~3,600 OpenAI calls (gpt-4o-mini + gpt-4o), disk-cached, resumable
python analyze.py   # per-model metrics, report.txt, and figures + cross-model comparison

run.py is safe to interrupt and rerun: every call is cached by (model, prompt, temperature), so only missing calls are re-issued. A full run costs a few dollars and takes under 10 minutes.

To rebuild the LaTeX paper:

cd report
pdflatex report.tex
pdflatex report.tex   # second pass resolves references

What this is not

We make no claim about whether a model "really has" these preferences, or about model welfare. This is a measurement reliability framework, not a consciousness or moral-status argument. See report.md §7 (Limitations) and §6 (Deferred work) for what we explicitly did not test, including a common-mode bias ablation that is the natural next step.

Citation

Naman Omar. "When Should You Trust an LLM's Preference? A PREF-VALID Case Study." 2026.