Repository navigation
v1.0.2
tagged this
22 Sep 11:52
On Blackwell with torch 2.14 / CUDA 13 the cuDNN scaled-dot-product-attention backend returns wrong, finite output for the masked rectangular attention that Engine.score_shared and the schema cache run (a suffix scored against a cached prefix). The shipped server answered Decision Index row RouterBench-5shot:26878 with a wrong option at p=0.93 and changed its answer between identical requests; the full forward and the corrected cached path give option_4 at p=0.95 / 0.93. Engine now calls set_attention_backend_policy() before compile and graph capture. Found by an independent review of the serving path. Test: tests/test_attention_policy.py. CI: a second job installs CPU torch, fastapi and httpx so the torch-dependent tests run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>