Skip to content

Make the LLM max-token request field configurable#7

Merged
missuo merged 1 commit into
missuo:mainfrom
hyspace:codex/llm-max-token-parameter
Mar 25, 2026
Merged

Make the LLM max-token request field configurable#7
missuo merged 1 commit into
missuo:mainfrom
hyspace:codex/llm-max-token-parameter

Conversation

@hyspace

@hyspace hyspace commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Summary

This change makes the max-token request field configurable for the OpenAI-compatible LLM integration.

Motivation

The known failure that prompted this change was observed with Azure AI Foundry, where requests fail when the max-token field does not match the target model's API expectations.

Azure's guidance for this issue indicates that some deployments require max_completion_tokens instead of max_tokens:
https://learn.microsoft.com/en-us/answers/questions/2139738/openai-badrequesterror-error-code-400-((error-((me

Making this field configurable keeps Koe compatible with endpoints that require a different parameter name, while preserving the current default behavior.

Changes

  • add a max_token_parameter config option under llm
  • support max_tokens and max_completion_tokens when building chat completion requests
  • keep the default value as max_tokens for backward compatibility with existing setups
  • document the new option in the sample config and README

Testing

  • cargo check

@hyspace hyspace changed the title Add configurable max token parameter for LLM requests Make the LLM max-token request field configurable Mar 25, 2026
@hyspace

hyspace commented Mar 25, 2026

Copy link
Copy Markdown
Contributor Author

noticed new requirements for contributing, updating PR

@hyspace hyspace force-pushed the codex/llm-max-token-parameter branch from b43cf5a to 2d2ccb2 Compare March 25, 2026 06:54
@missuo

missuo commented Mar 25, 2026

Copy link
Copy Markdown
Owner

noticed new requirements for contributing, updating PR

You just need to modify the commit message, there are no other major problems.

@missuo missuo merged commit 43e3d53 into missuo:main Mar 25, 2026
@hyspace hyspace deleted the codex/llm-max-token-parameter branch March 29, 2026 04:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants