Skip to content

Release v0.22.0

Choose a tag to compare

@github-actions github-actions released this 20 Oct 10:06
· 14 commits to main since this release

Added

  • EvaluationOptions.Thinking — the reasoning mode of every evaluator's verdict call (Faithfulness, Relevancy,
    Answerability, AnswerCoverage), passed to CompletionOptions.Thinking.
  • OpenAICompatibleCompletionService.ReasoningSwitch — how CompletionOptions.Thinking reaches an OpenAI-compatible server.
    The service ignored Thinking, so the Off that chunk enrichment and (now) the evaluators ask for never reached a reasoning
    model on such a server. OpenAICompatibleReasoningSwitch.ChatTemplateKwargs sends chat_template_kwargs
    (enable_thinking / thinking) for On / Off — vLLM, SGLang, llama.cpp (GPUStack included); None (the default) sends
    nothing, which keeps OpenAI itself working. Also AddFluxImproverWithOpenAI(…, reasoningSwitch, …).

Changed

  • Breaking — evaluators ask for no reasoning by default (ThinkingMode.Off). They sent no thinking mode, so a grader
    that reasons on its default could spend MaxTokens on hidden reasoning and return no verdict — a scoring failure, not a
    score. Same judgement as EnrichmentOptions.Thinking. Migration: to keep the model's own default, set
    new EvaluationOptions { Thinking = ThinkingMode.Auto } (and raise MaxTokens if it reasons).

Dependencies

  • Re-pinned sibling package(s) LMSupply.Generator 0.128.0 -> 0.129.0, LMSupply.Generator.Onnx 0.128.0 -> 0.129.0.
  • Re-pinned sibling package(s) LMSupply.Generator 0.129.0 -> 0.131.0, LMSupply.Generator.Onnx 0.129.0 -> 0.131.0.