You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
EvaluationOptions.Thinking — the reasoning mode of every evaluator's verdict call (Faithfulness, Relevancy,
Answerability, AnswerCoverage), passed to CompletionOptions.Thinking.
OpenAICompatibleCompletionService.ReasoningSwitch — how CompletionOptions.Thinking reaches an OpenAI-compatible server.
The service ignored Thinking, so the Off that chunk enrichment and (now) the evaluators ask for never reached a reasoning
model on such a server. OpenAICompatibleReasoningSwitch.ChatTemplateKwargs sends chat_template_kwargs
(enable_thinking / thinking) for On / Off — vLLM, SGLang, llama.cpp (GPUStack included); None (the default) sends
nothing, which keeps OpenAI itself working. Also AddFluxImproverWithOpenAI(…, reasoningSwitch, …).
Changed
Breaking — evaluators ask for no reasoning by default (ThinkingMode.Off). They sent no thinking mode, so a grader
that reasons on its default could spend MaxTokens on hidden reasoning and return no verdict — a scoring failure, not a
score. Same judgement as EnrichmentOptions.Thinking. Migration: to keep the model's own default, set new EvaluationOptions { Thinking = ThinkingMode.Auto } (and raise MaxTokens if it reasons).