Amazon bedrock - wrong limits

[fixerivan]

App Version

Version: 3.17.2

API Provider

AWS Bedrock

Model Used

arn:aws:bedrock:us-east-1::inference-profile/us.anthropic.claude-3-7-sonnet-20250219-v1:0

🔁 Steps to Reproduce

when i use custom ARN profile in AWS bedrock the model parameters are not correctly read, the max context size is set to 128k even though the model is claude 3.7 with 200k context, also the Max output is set to 5,000 tokens

there is no way to manually correct these values

💥 Outcome Summary (Optional)

No response

📄 Relevant Logs or Errors

[adamhill]

Here is the patch. Its needs to be tested with Bedrock credentials.

// PATCH: update getModelById to honor user overrides for contextWindow and maxTokens

getModelById(modelId: string, modelType?: string): { id: BedrockModelId | string; info: SharedModelInfo } {
	// Try to find the model in bedrockModels
	const baseModelId = this.parseBaseModelId(modelId) as BedrockModelId;

	let model;
	if (baseModelId in bedrockModels) {
		// Deep copy
		model = { id: baseModelId, info: JSON.parse(JSON.stringify(bedrockModels[baseModelId])) };
	} else if (modelType && modelType.includes("router")) {
		model = {
			id: bedrockDefaultPromptRouterModelId,
			info: JSON.parse(JSON.stringify(bedrockModels[bedrockDefaultPromptRouterModelId])),
		};
	} else {
		// PATCH: Allow user overrides for modelMaxTokens and modelContextWindow when not found
		model = {
			id: bedrockDefaultModelId,
			info: {
				...JSON.parse(JSON.stringify(bedrockModels[bedrockDefaultModelId])),
				// Use user-specified overrides if provided, else fallback to default
				maxTokens: this.options.modelMaxTokens && this.options.modelMaxTokens > 0
					? this.options.modelMaxTokens
					: 4096,
				contextWindow: this.options.modelContextWindow && this.options.modelContextWindow > 0
					? this.options.modelContextWindow
					: 128000,
			},
		};
	}

	// If modelMaxTokens is explicitly set in options, override the default
	if (this.options.modelMaxTokens && this.options.modelMaxTokens > 0) {
		model.info.maxTokens = this.options.modelMaxTokens;
	}
	// PATCH: Allow contextWindow to also be overridden from options
	if (this.options.modelContextWindow && this.options.modelContextWindow > 0) {
		model.info.contextWindow = this.options.modelContextWindow;
	}

	return model;
}