-ffast-math on the HIP backend breaks greedy speculative decoding (output no longer matches non-speculative greedy)

[jimw567]

PR #23862 introduced fast-math flag on hip backend and made it the default and only option. It improves the speed by ~%6 as shown in the PR, but it breaks a fundamental contract that MTP self-speculation at temperature=0 must yield output that is byte-for-byte identical to the non-speculative baseline.

Reproduction
Run the same greedy prompt twice against llama-server, once without speculation and once with MTP self-speculation, and diff the completions:

baseline

llama-server -m Qwen3.5-4B-Q4_0.gguf -ngl 999 -fa on -c 4096 --spec-type none

speculative

llama-server -m Qwen3.5-4B-Q4_0.gguf -ngl 999 -fa on -c 4096 --spec-type draft-mtp --spec-draft-n-max 3

I respectably request that PR#23862 to be reverted or guarded with an option and off by default.

CC @a-huk @am17an @IMbackK

[jimw567]

OK. PR #23862 was essentially reverted by PR #25495. However, PR #25495 replaced fast-math with unsafe-math, which still carries the issue. I raised the PR below to make unsafe-math guarded by an option with default value OFF. IEEE-conformant is more important than speed IMHO. End users will need to make a conscious decision to trade accuracy and reproducibility with speed.

#26696