ratio: is a proportional-reasoning problem hard because of the numbers' size or a fractional unit price?

A one-step ratio word problem: "If A items cost B dollars, how much do C items cost?" The answer is always the exact integer B*C/A. Two things can make it hard for a small model:

• number magnitude - how large the answer is, and
• a fractional unit price - whether B/A is a clean integer (5 items for 15 dollars -> 3 each) or a fraction (6 items for 15 dollars -> 2.5 each), even though the final answer stays an integer.

The question: which one actually drives difficulty, and does it depend on model size? Two models, Qwen2.5-1.5B-Instruct and Qwen2.5-0.5B-Instruct (Q4_K_M), greedy, n=80 per unit-price kind.

Pre-registered prediction (written before the controlled run)

A fractional unit price is harder than an integer one for both models, because it forces an intermediate that is not a whole number, and the effect is larger for the smaller model.

What actually happened: the prediction was falsified for the larger model, and a confound had to be fixed first

The first version of the item generator produced smaller numbers for the fractional problems than for the integer ones. Under it the 1.5B scored higher on fractional problems - the opposite of the prediction - because they were simply smaller-number problems. That is a magnitude confound, not evidence against the hypothesis. The generator was widened so both kinds span the same answer magnitudes, and accuracy is reported bucketed by answer size (small <=30, large >30) so the two effects are separated.

With magnitude controlled:

model	small (answer <=30)	large (>30)	small: integer vs fractional	large: integer vs fractional
1.5B	0.70	0.45	0.71 vs 0.69 (frac-int -0.03)	0.44 vs 0.45 (+0.00)
0.5B	0.16	0.08	0.31 vs 0.06 (frac-int -0.26)	0.09 vs 0.07 (-0.02)

Three findings:

1. Number magnitude is the dominant driver for both models. Accuracy roughly halves from small to large answers (1.5B 0.70 -> 0.45, 0.5B 0.16 -> 0.08). This is the effect the pre-registration missed.
2. A fractional unit price is a non-issue for the 1.5B. At matched magnitude its accuracy is the same whether the unit price is a whole number or a fraction (frac-minus-integer -0.03 small, +0.00 large). The pre-registered "fractional is harder" is falsified for this model.
3. The same fractional unit price crushes the 0.5B where it has any ability: at small magnitude accuracy falls from 0.31 (integer) to 0.06 (fractional). On large answers both are near floor. So the fractionality penalty is capability-gated - it bites only below a threshold the 1.5B has already crossed.

Wrong answers are genuine miscalculations, not surface copying: only 6.0% (1.5B) and 7.9% (0.5B) of errors echo one of the input numbers A, B, or C.

Why this is the honest read

The clean single-number story - "fractions are hard" - is wrong for the capable model and right for the weak one, and neither is visible until magnitude is held fixed. Externalizing an awkward intermediate (a non-integer unit price) only costs accuracy when the model is already near its arithmetic limit; a more capable model absorbs it. What both models actually pay for is bigger numbers.

Reproduce

./reproduce.sh 8081 8082        # PORT_15B PORT_05B
./scripts/gate.sh               # ruff, mypy --strict, pytest, ASCII, verify

tools/verify.py is an independent recompute: it re-reads the raw predictions, re-derives every answer from its own B*C/A oracle, re-buckets by magnitude, and asserts the three findings. It shares no code with src/ or tools/analyze.py.

Limitations and falsifiers

• Two model sizes, one task template, one phrasing, greedy decoding, n=80 per unit-price kind (n=160 per model). Not a claim about larger models, chain-of-thought prompting, or other tasks.
• Magnitude and fractionality are separated by bucketing at a single cut (answer=30), not modelled jointly; the qualitative picture (magnitude dominates, fractionality is capability-gated) is what is claimed, not the exact per-cell numbers.
• The 0.5B large-magnitude cells are near floor, so the small fractionality effect there is not over-read.
• Falsifier: if, at matched magnitude, the 1.5B's fractional accuracy were materially below its integer accuracy (frac-minus-integer worse than -0.1 in either bucket), finding 2 would be wrong.
• Falsifier: if the small-to-large magnitude drop vanished for either model, finding 1 would be wrong.

MIT licensed. Ground truth is exact integer arithmetic; no judge, no LLM grading.