Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure 'name' on initial message #2635

Merged
merged 11 commits into from
Aug 20, 2024
Merged

Ensure 'name' on initial message #2635

merged 11 commits into from
Aug 20, 2024

Conversation

marklysze
Copy link
Collaborator

@marklysze marklysze commented May 9, 2024

Why are these changes needed?

When initiating a chat through ConversableAgent's initiate_chat the passed in message in the conversation doesn't get the name of the agent initiating the conversation attached to it. This is, then, not passed through to the LLM and it cannot use that name information.

So, running this block of code:

cathy = ConversableAgent(
    "cathy",
    system_message="Your name is Cathy and you are a part of a duo of comedians.",
    llm_config={"config_list": config_list, "cache_seed": None},
    human_input_mode="NEVER",  # Never ask for human input.
)

joe = ConversableAgent(
    "joe",
    system_message="Your name is Joe and you are a part of a duo of comedians.",
    llm_config={"config_list": config_list, "cache_seed": None},
    human_input_mode="NEVER",  # Never ask for human input.
)

result = joe.initiate_chat(cathy, message="Tell me a joke and include both of our names in it!", max_turns=2)

will result in a list of messages to the LLM like this:

"[
{'content': 'Your name is Cathy and you are a part of a duo of comedians.', 'role': 'system'},
{'content': 'Tell me a joke and include both of our names in it!', 'role': 'user'}
]"

I would expect there to be a name key, with the value joe, attached to the second message as that agent initiated the chat.

The response from the LLM is:
'Why did Cathy and her comedy partner decide to open a bakery together?\n\nBecause they realized they "dough"n\'t just make people laugh, they can also make them delicious pastries!'

... and it can be seen that it doesn't reference Joe's name.

By including the name key/value in the messages, like this:

"[
{'content': 'Your name is Cathy and you are a part of a duo of comedians.', 'role': 'system'},
{'content': 'Tell me a joke and include both of our names in it!', 'role': 'user', 'name': 'joe'}]"

... the LLM can use the name and returns this:
'Why did Cathy and Joe go to the comedy club?\n\nBecause they heard it was a great place for two funny people to crack jokes and make people laugh!'

... indicating that the name field is utilised (and LLMs aren't great at jokes!).

To fix this, the ConversableAgent's _append_oai_message has been updated to add the name field, if it doesn't exist and is not a function/tool message. To support this addition, an is_sending parameter is used on the function to indicate whether the self agent or conversation_id agent is the sender and, hence, the name to attach to the message.

I have not added this to documentation or updated tests for this. Please let me know if it needs specific tests added.

Related issue number

No related issue.

Checks

@qingyun-wu qingyun-wu added this pull request to the merge queue Aug 20, 2024
Merged via the queue into main with commit 77ae3c0 Aug 20, 2024
134 of 146 checks passed
@qingyun-wu qingyun-wu deleted the appendnameinitialmsg branch August 20, 2024 04:14
open-autogen pushed a commit to autogenhub/autogen that referenced this pull request Aug 21, 2024
* Update to ensure name on initial messages

* Corrected test cases for messages now including names.

* Added name to messages within select speaker nested chat

* Corrected select speaker group chat tests for name field

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
randombet pushed a commit to randombet/autogen that referenced this pull request Aug 21, 2024
* Update to ensure name on initial messages

* Corrected test cases for messages now including names.

* Added name to messages within select speaker nested chat

* Corrected select speaker group chat tests for name field

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
victordibia pushed a commit that referenced this pull request Aug 28, 2024
* Update to ensure name on initial messages

* Corrected test cases for messages now including names.

* Added name to messages within select speaker nested chat

* Corrected select speaker group chat tests for name field

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants