Skip to content

v0.1.4: hosted temperature-0 check

Choose a tag to compare

@rohithreddybc rohithreddybc released this 26 Sep 03:49

Adds a hosted-endpoint check: openai/gpt-oss-20b on Groq at temperature 0 (five replicates) still disagreed with itself in 22 of 960 comparisons, across 48 distinct system fingerprints. Adds groq_answerer.py and selects answerers with FMA_ANSWERER. The verifier checks 188 claims. CITATION.cff cites the manuscript Instability Floors: Separating Bias from Noise in Fairness Audits of Clinical LLM Agents with FairMedAgent. The supplementary material is attached. Commit a6fd6d6.