Multiple competions (n)

[svilupp]

• Initial support for multiple completions in one request for OpenAI-compatible API servers. Set via API kwarg n=5 and it will request 5 completions in one request, saving the network communication time and paying the prompt tokens only once. It's useful for majority voting, diversity, or challenging agentic workflows.
• Added new fields to AIMessage and DataMessage types to simplify tracking in complex applications. Added fields:
  • cost - the cost of the query (summary per call, so count only once if you requested multiple completions in one call)
  • log_prob - summary log probability of the generated sequence, set API kwarg logprobs=true to receive it
  • run_id - ID of the AI API call
  • sample_id - ID of the sample in the batch if you requested multiple completions, otherwise sample_id==nothing (they will have the same run_id)
  • finish_reason - the reason why the AI stopped generating the sequence (eg, "stop", "length") to provide more visibility for the user

[codecov]

Codecov Report

Attention: Patch coverage is 97.77778% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 92.27%. Comparing base (84f1054) to head (320fdc5).

Files	Patch %	Lines
src/llm_openai.jl	96.55%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #79      +/-   ##
==========================================
+ Coverage   92.02%   92.27%   +0.25%     
==========================================
  Files          36       36              
  Lines        1818     1890      +72     
==========================================
+ Hits         1673     1744      +71     
- Misses        145      146       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.