Azure OpenAI API requires 'content' field despite documentation stating otherwise

[pachacamac]

Description:
When interacting with the Azure OpenAI API, the 'content' field is mandated even though the official API documentation mentions it as optional. This inconsistency leads to an error message: 'content' is a required property - 'messages.2'.

Steps to Reproduce:

1. Set up Azure OpenAI API integration as per the documentation.
2. Make a request without the 'content' field.
3. Observe the error message.

Expected Behavior:
The API should accept requests without the 'content' field, as per the documentation.

Actual Behavior:
The API returns an error message: 'content' is a required property - 'messages.2'.

Reference:
Error can be traced back to conversible_agent.py:274.

Environment:

• Azure OpenAI version: 2023-08-01-preview

[sonichi]

Transferring this to the autogen repo. What did you use in step 2?

[pachacamac]

It's a standard function call request where content is absent as described in OpenAI docs.

FYI overriding ConversibleAgent._append_oai_message with this, fixes the Error for me.

def _append_oai_message(self, message: Union[Dict, str], role, conversation_id: Agent) -> bool:
        """Append a message to the ChatCompletion conversation.

        If the message received is a string, it will be put in the "content" field of the new dictionary.
        If the message received is a dictionary but does not have any of the two fields "content" or "function_call",
            this message is not a valid ChatCompletion message.

        Args:
            message (dict or str): message to be appended to the ChatCompletion conversation.
            role (str): role of the message, can be "assistant" or "function".
            conversation_id (Agent): id of the conversation, should be the recipient or sender.

        Returns:
            bool: whether the message is appended to the ChatCompletion conversation.
        """
        message = self._message_to_dict(message)
        # create oai message to be appended to the oai conversation that can be passed to oai directly.
        oai_message = {k: message[k] for k in ("content", "function_call", "name", "context") if k in message}
        if "content" not in oai_message and "function_call" not in oai_message:
            return False
        
        oai_message["role"] = "function" if message.get("role") == "function" else role

        # XXX: Adding empty content to satisfy wrong api spec
        if oai_message["role"] == "assistant" and "function_call" in oai_message and not "content" in oai_message:
            oai_message["content"] = ""

        self._oai_messages[conversation_id].append(oai_message)
        return True

[sonichi]

Thanks. @kevin666aa is this a known issue to you?

[yiranwu0]

Thank you! We are fixing this in PR #47. We will set "content" to None if only "function_call" is in the message. This error might occur because the model make a function call directly.

With OpenAI API, it is observed that content is set to None.

# returned message
{
"content" : None,
"function_call":  "..."
...
}

It might be different with AzureAI (That the field is not there?), and we will do the same handling of setting it to None.

[yiranwu0]

@pachacamac We updated the code. Can you pull and test it to see if the error persists?

[pachacamac]

Hey @kevin666aa @sonichi thank you for the quick turnaround! The fix works :)