Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] chat_agent update message #298

Closed
2 of 3 tasks
zhiyu-01 opened this issue Sep 20, 2023 · 4 comments · Fixed by #302
Closed
2 of 3 tasks

[BUG] chat_agent update message #298

zhiyu-01 opened this issue Sep 20, 2023 · 4 comments · Fixed by #302
Assignees
Labels
bug Something isn't working

Comments

@zhiyu-01
Copy link
Contributor

Required prerequisites

What version of camel are you using?

0.1.0

System information

3.10.12 | packaged by Anaconda, Inc. | (main, Jul 5 2023, 19:01:18) [MSC v.1916 64 bit (AMD64)] win32
0.1.0

Problem description

I have added a function web_search into camel recently, but it didn't work correctly:
image
The user can't provide instructions, but the assistant can, which is against the prompt.

In chat_agent.py , there are these codes:
image
When agent update message, it always named 'user' even it's an assistant agent. And the same as submit_message function:
image
The message comes from assistant have been updated to user's, I guess this is the reason why agent got confused, but it
didn't confuse agents in other conversation types.

In role_playing.py init_chat function, there is also a strange code:
image
The assistant message given to user, and user message given to assistant. But here the assistant_response = self.assistant_agent.step(user_msg) haven't been used, it's just update the message to assistant agent's stored_messages list, in most example files, the init_chat just used in this way:
image
The response from the agent.step have been throwed away. More important, it generates a message with user role type and assistant content:
image
The init_chat can work correctly in this way:
image

I have created a test with two steps of role_playing to output the agent's stored_messages:
image

here I only printed the role type,
This is the original code result:
image
This is the modified code result:
image

The conversation runs alternate in two agents, shouldn't the messages store in the same way?

After these modify, the agent back to normal:
image

Reproducible example code

python role_playing_with_function.py

Traceback

No response

Expected behavior

No response

Additional context

No response

@zhiyu-01 zhiyu-01 added the bug Something isn't working label Sep 20, 2023
@zhiyu-01 zhiyu-01 mentioned this issue Sep 20, 2023
13 tasks
@Obs01ete
Copy link
Collaborator

I think the misunderstanding comes from the following.

There are two AI agents:

  1. AI user
  2. AI assistant

And there are two orthogonal "roles at backend":

  1. user
  2. assistant
    which are related to OpenAI API.

This is quite untrivial and the naming is not ideal, I agree. Please take this into account and update us if you think that the bug is still there.

@lightaime

@Obs01ete
Copy link
Collaborator

Anyway, @zhiyu-01 please create a PR for this ticket and put the test that you made earlier here.

@zhiyu-01
Copy link
Contributor Author

I have printed out the user's ChatRecord, it should be in sequence as ['system', 'assistant', 'user'], but we just get a chaos:
user

The intersting thing is the agent didn't get confused in other role plyaing example, even the chatrecord is also out of order:
image

Besides, the ChatRecord containts two properties, role_at_back_end and message, the message is also containts the role type and content properties, what openai api need is {'role': role, 'content': content}, here seems have a little redundant. Should we consider a chage in the ChatRecord class?
@Obs01ete

@Obs01ete
Copy link
Collaborator

@zhiyu-01 Please take the test

def test_role_playing_role_sequence(model_type=None):

and put it into a new PR for this ticket. Make sure that the test checks for the following sequence:

[lightaime](https://github.com/lightaime) [yesterday](https://github.com/camel-ai/camel/issues/298#r1335667054)
I have some doubts about this test. Should it be ['system', 'user', 'assistant', 'user', 'assistant', ...] normally?

@lightaime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants