Repository navigation
Patch on v2.7.0.
Fix: gemini-3.8-flash rejects thinking_level=minimal with a 400, exactly as 3.7 does, but the denylist named only 3.7. Pointing model: at 3.8 would have hard-failed every chunked transcript. Added to _NO_MINIMAL_THINKING_LEVEL, to model_eval.py's defaults and pricing, and the test is parametrized over both models (#219, 0436f6a).
Measured, not adopted: gemini-3.8-flash vs the 3.7 incumbent, three rolls each on two new fixture shapes (four-presenter panel, busy screen with two presenters). 3.7 kept all four speakers on every roll and held a lower median max gap; 3.8 dropped the speakers and screen_content tasks once in three. The default stays gemini-3.7-flash. Card: tests/evals/model-cards/gemini-3.8-flash.md. README gained a short "newer model is not automatically better" note.
Two fixture shapes were added to tests/evals/model_fixtures.yaml.example.