Bug: ChatHistory management is not supported for thinking model

[HuskyDanny]

I am using AzureChatCompletion as agent client which uses o3-mini to use function calling. Since I am using Service selector, which could move from 4o to o3-mini, the inconsistency of the two models is the root cause, but SK didn't handle automatic convert system message to developer message for thinking models, leading unexpected behavior. I thought this is the fundamental issue and surprised no one raised it, are there any workarounds or best practices for this?

For ChatCompletionAgent class:

I think it might need to check what the service selector returns, and to decide to add developer message or system message.

[moonbox3]

Hi @HuskyDanny, the conversion happens deeper in the stack, right before calling the model. On this line, you can see we check if the instruction_role is set to developer then convert any system messages to be that of developer.

semantic-kernel/python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

Lines 294 to 303 in 6654376

	return [
	{
	**message.to_dict(role_key=role_key, content_key=content_key),
	role_key: "developer"
	if self.instruction_role == "developer" and message.to_dict(role_key=role_key)[role_key] == "system"
	else message.to_dict(role_key=role_key)[role_key],
	}
	for message in chat_history.messages
	if not isinstance(message, (AnnotationContent, FileReferenceContent))
	]

I just noticed that this hasn't made it into our learn docs yet, so filing an issue to track adding that in. It is present, "hidden" in a concept sample:

semantic-kernel/python/samples/concepts/reasoning/simple_reasoning.py

Line 59 in 6654376

chat_service = OpenAIChatCompletion(service_id="reasoning", instruction_role="developer")

Of course, one can switch AzureChatCompetion for the OpenAI AI service shown there.

[HuskyDanny]

@moonbox3 Got it, thank you!

[moonbox3]

You're welcome, @HuskyDanny. I will close this issue. Let us know if you have any other questions.