feat: add Anthropic support #288

ocss884 · 2023-09-14T06:29:19Z

Description

Describe your changes in detail.

Motivation and Context

close #283

I have raised an issue to propose this change (required for new features and bug fixes)

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds core functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation (update in the documentation)
Example (update in the folder of example)

Implemented Tasks

add Anthropic LLM (Claude-2, Claue-instant-1) backend
add an example of using Claude

Checklist

Go over all the following points, and put an x in all the boxes that apply.
If you are unsure about any of these, don't hesitate to ask. We are here to help!

I have read the CONTRIBUTION guide. (required)
My change requires a change to the documentation.
I have updated the tests accordingly. (required for a bug fix or a new feature)
I have updated the documentation accordingly.

Obs01ete

Thank you @ocss884. This is a very well prepared PR. Let's sort out what is wrong with the two system messages in the main code. Also we will need tests for the code that you added including examples. Please wrap the code that would crash if not Anthropic key provided into skipif as here:

camel/test/models/test_open_source_model.py

Line 61 in 3ba1754

@pytest.mark.skipif(not check_server_running(DEFAULT_SERVER_URL),

Please run code coverage as in CONTRIBUTING.md and for all the code that you have added make sure that it covered.

Obs01ete · 2023-10-08T11:50:23Z

camel/utils/token_counting.py

+        ret = system_prompt
+        for msg in messages[1:]:
+            role, content = msg["role"], msg["content"]
+            # Claude does not perform well if the system message RULE OF USER/ASSISTANT is sent twice. (see role_playing/RolePlaying.init_chat())


Thank you for spotting. Did you discover a bug of sending the system prompt twice? We never observed it. Could you create a bug ticket to reproduce it and fix? We'd better fix the erroneous behavior in the main code than create a detour here in your code.

I think it is an expected behavior since in init_chat() the doc_string says "..sending the system message again to the agents using chat messages". The full doc_string is as below:

camel/camel/societies/role_playing.py

Lines 351 to 355 in 3ba1754

def init_chat(self) -> Tuple[BaseMessage, List[BaseMessage]]:

r"""Initializes the chat by resetting both of the assistant and user

agents, and sending the system messages again to the agents using

chat messages. Returns the assistant's introductory message and the

user's response messages.

I should say it is confusing for me since in role_play, this init_chat function is used to get the first input_assistant_msg and often it is just a modified version of user_sys_msg. Look at the first return value:

camel/camel/societies/role_playing.py

Lines 366 to 379 in 3ba1754

assistant_msg = BaseMessage.make_assistant_message(

role_name=self.assistant_sys_msg.role_name,

content=(f"{self.user_sys_msg.content}. "

"Now start to give me instructions one by one. "

"Only reply with Instruction and Input."))

user_msg = BaseMessage.make_user_message(

role_name=self.user_sys_msg.role_name,

content=f"{self.assistant_sys_msg.content}")

assistant_response = self.assistant_agent.step(user_msg)

if assistant_response.terminated or assistant_response.msgs is None:

raise ValueError(f"Assistant agent terminated unexpectedly. "

f"Error info: {assistant_response.info}")

return assistant_msg, assistant_response.msgs

For example in the open_source_example, it is used as input_assistant_msg

camel/examples/open_source_models/role_playing_with_open_source_model.py

Line 60 in 3ba1754

input_assistant_msg, _ = role_play_session.init_chat()

and then this modified user_sys_msg is send to usr once again during the first round of chatting

camel/camel/societies/role_playing.py

Line 433 in 3ba1754

user_response = self.user_agent.step(assistant_msg)

So during the first API call, the assistant_msg is just the whole user_sys_msg append with two more instructions:

camel/camel/societies/role_playing.py

Lines 366 to 370 in 3ba1754

assistant_msg = BaseMessage.make_assistant_message(

role_name=self.assistant_sys_msg.role_name,

content=(f"{self.user_sys_msg.content}. "

"Now start to give me instructions one by one. "

"Only reply with Instruction and Input."))

That makes what I find a "duplicate system messages". I'm happy to create a ticket and solve it if you can confirm this is not desired.

For me, I would prefer to make

camel/camel/societies/role_playing.py

Lines 369 to 370 in 3ba1754

"Now start to give me instructions one by one. "

"Only reply with Instruction and Input."))

only as the very first assistant_msg. It is more intuitive to send the rule once and then instruct the user_model to go first using simple instructions. Also, I recommend adding more comments to init_chat if you want to keep it. It is confusing of making a variable called assistant_msg with the content of usr_sys_msg.

dandansamax

Great code! Leave some comments.

camel/utils/token_counting.py

examples/anthropic_models/role_playing_with_claude.py

pyproject.toml

camel/configs.py

ocss884 · 2024-01-03T05:35:27Z

Anthropic have a more openai-like 'Message completion' in beta now to replace their current text completion.
In the new function, people can use the user-assistant form, for example:

[
  {"role": "user", "content": "Hello there."},
  {"role": "assistant", "content": "Hi, I'm Claude. How can I help you?"},
  {"role": "user", "content": "Can you explain LLMs in plain English?"},
]

I will update once they are done.

Wendong-Fan · 2024-03-14T16:11:43Z

Hey @ocss884 , I plan to do the review later this week, would you like to solve the conflicts and update this PR with latest master branch? Thanks a lot!

coderabbitai · 2024-03-17T21:29:55Z

Important

Auto Review Skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai generate interesting stats about this repository and render them as a table.
- @coderabbitai show all the console.log statements in this repository.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

ocss884 · 2024-03-17T23:23:17Z

Hey @ocss884 , I plan to do the review later this week, would you like to solve the conflicts and update this PR with latest master branch? Thanks a lot!

Hi @Wendong-Fan I just updated this PR. Due to an openai recent release, I limit the package version #474 . Some test cases are failed cuz this PR was opened when I was not a member and that makes the OPENAI_API_KEY secret empty (you can check the log). pytest . gives an all pass on my local.

Wendong-Fan

Thanks for the contribution, overall looks great, left some comments, after your update we can merge it

camel/utils/commons.py

pyproject.toml

camel/models/anthropic_model.py

Wendong-Fan

Thanks for the contribution! Please feel free to resolve the conflicts and merge it.

ocss884 · 2024-04-01T16:18:56Z

Going to add Claude3 support before merge it.

ocss884 · 2024-04-17T07:13:35Z

@Wendong-Fan @dandansamax Hi Wendong and Tianqi. Claude3 is ready now. But it seems I cannot approve the PR myself. @dandansamax could you help take a look? Thanks!

This reverts commit 25c8e42.

Appointat · 2024-04-21T21:06:25Z

Hi, @ocss884 , I found this PR was merged without approval and the pipelines were not passed. It is the wrong approach, and it must be reverted.

Hi, @Wendong-Fan . You have to see this, please. ❗

And I created a PR to revert the last commit in master branch. #518

lightaime · 2024-05-25T17:36:18Z

camel/utils/token_counting.py

+        Returns:
+            int: Number of tokens in the messages.
+        """
+        prompt = messages_to_prompt(messages, self.model_type)


It seems messages_to_prompt does not support anthropic models yet. How does it work?

camel/camel/utils/token_counting.py

Line 39 in 916ab5e

def messages_to_prompt(messages: List[OpenAIMessage], model: ModelType) -> str:

Wendong-Fan · 2024-05-26T04:51:46Z

camel/utils/token_counting.py

-    elif model.is_anthropic:
-
-        # use XML tag to decorate prompt
-        # https://docs.anthropic.com/claude/docs/constructing-a-prompt#mark-different-parts-of-the-prompt
-        # https://docs.anthropic.com/claude/docs/roleplay-dialogue
-        # https://docs.anthropic.com/claude/docs/human-and-assistant-formatting
-        system_message = str(system_message)
-        ret = f"\n{system_message}\n"
-        for msg in messages[1:]:
-            role, content = msg["role"], msg["content"]
-            # Claude does not perform well if the system message RULE OF USER/ASSISTANT is sent twice. (see role_playing/RolePlaying.init_chat())
-            # Here is a special treatment for Claude to remove the redundant system message.
-            if not isinstance(content, str):
-                raise ValueError("Currently multimodal context is not "
-                                 "supported by the token counter.")
-            if content.startswith(system_message):
-                ret = content
-                continue
-            if role == "user":
-                ret += f"{HUMAN_PROMPT} {content}"
-            elif role == "assisstant":
-                ret += f"{AI_PROMPT} {content}"
-        return ret
    else:


why did this part deleted? @ocss884

cc @lightaime

ocss884 added 4 commits September 13, 2023 23:16

add Anthropic support camel-ai#283

a1b9b28

correct lint and format

b242cc0

refresh poetry.lock

201a4e2

Use union instead of "|" for py3.8

de6d6ee

Benjamin-eecs changed the title ~~add Anthropic support #283~~ feat: add Anthropic support Sep 18, 2023

HalberdOfPineapple requested review from Obs01ete, HalberdOfPineapple and dandansamax and removed request for Obs01ete and HalberdOfPineapple September 20, 2023 08:39

Obs01ete assigned ocss884 Oct 5, 2023

Create pull.yml

c3265f8

Obs01ete requested changes Oct 8, 2023

View reviewed changes

dandansamax requested changes Nov 5, 2023

View reviewed changes

ocss884 and others added 7 commits November 10, 2023 10:33

Merge branch 'camel-ai:master' into master

e735d9a

Merge branch 'camel-ai:master' into master

c0e55ea

Merge branch 'camel-ai:master' into master

35b3673

update to "master"

808b2cf

apply camel-ai#288, migrate to openai v1.0.0

ff6f94c

update lock file and correct code format

e597c05

Merge branch 'master' into anthropic_support

4fde44a

ocss884 requested a review from dandansamax December 27, 2023 00:17

bug fix in commons.py

1f85339

ocss884 closed this Jan 3, 2024

ocss884 reopened this Jan 3, 2024

ocss884 mentioned this pull request Jan 3, 2024

[BUG] fix duplicate system message #421

Closed

3 tasks

update test file and remove mistakenly tracked pull.yml

72bfdef

ocss884 mentioned this pull request Mar 7, 2024

[Feature Request] Integrate Claude 3 #455

Closed

2 tasks

Wendong-Fan self-requested a review March 8, 2024 13:21

ocss884 added 2 commits March 17, 2024 14:17

Merge remote-tracking branch 'upstream/master'

faec388

Merge branch 'master' into anthropic_support

fcf4511

ocss884 added 2 commits March 17, 2024 14:33

update lock file

fe368e1

limit openai version<1.14.0

9dd1822

Wendong-Fan reviewed Mar 23, 2024

View reviewed changes

camel/utils/commons.py Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

pyproject.toml Outdated Show resolved Hide resolved

camel/models/anthropic_model.py Outdated Show resolved Hide resolved

add docstring and improve api_key checking logic

a48a57e

Wendong-Fan approved these changes Mar 29, 2024

View reviewed changes

Wendong-Fan linked an issue Apr 6, 2024 that may be closed by this pull request

[Feature Request] Integrate Claude 3 #455

Closed

2 tasks

ocss884 and others added 5 commits April 14, 2024 21:29

Merge branch 'master' of https://github.com/camel-ai/camel

2f5d17f

Merge branch 'master' into anthropic_support

156402e

add Claude3 support & migrate to Message endpoint

f202c7c

small fix

85bbb18

Merge branch 'master' into anthropic_support

1138868

ocss884 requested a review from a team April 17, 2024 07:11

Wendong-Fan added 2 commits April 17, 2024 23:40

Merge branch 'master' into anthropic_support

425a531

Merge branch 'master' into anthropic_support

cdc6744

ocss884 merged commit 25c8e42 into camel-ai:master Apr 21, 2024
1 of 6 checks passed

Appointat added a commit that referenced this pull request Apr 21, 2024

Revert "feat: add Anthropic support (#288)"

d410d7a

This reverts commit 25c8e42.

lightaime reviewed May 25, 2024

View reviewed changes

Wendong-Fan reviewed May 26, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add Anthropic support #288

feat: add Anthropic support #288

ocss884 commented Sep 14, 2023 •

edited by dandansamax

Loading

Obs01ete left a comment

Obs01ete Oct 8, 2023

ocss884 Oct 13, 2023 •

edited

Loading

dandansamax left a comment

ocss884 commented Jan 3, 2024

Wendong-Fan commented Mar 14, 2024

coderabbitai bot commented Mar 17, 2024 •

edited

Loading

Auto Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (`.coderabbit.yaml`)

Documentation and Community

ocss884 commented Mar 17, 2024

Wendong-Fan left a comment

Wendong-Fan left a comment

ocss884 commented Apr 1, 2024

ocss884 commented Apr 17, 2024

Appointat commented Apr 21, 2024 •

edited

Loading

lightaime May 25, 2024

Wendong-Fan May 26, 2024

	def init_chat(self) -> Tuple[BaseMessage, List[BaseMessage]]:
	r"""Initializes the chat by resetting both of the assistant and user
	agents, and sending the system messages again to the agents using
	chat messages. Returns the assistant's introductory message and the
	user's response messages.

	assistant_msg = BaseMessage.make_assistant_message(
	role_name=self.assistant_sys_msg.role_name,
	content=(f"{self.user_sys_msg.content}. "
	"Now start to give me instructions one by one. "
	"Only reply with Instruction and Input."))
	user_msg = BaseMessage.make_user_message(
	role_name=self.user_sys_msg.role_name,
	content=f"{self.assistant_sys_msg.content}")
	assistant_response = self.assistant_agent.step(user_msg)
	if assistant_response.terminated or assistant_response.msgs is None:
	raise ValueError(f"Assistant agent terminated unexpectedly. "
	f"Error info: {assistant_response.info}")

	return assistant_msg, assistant_response.msgs

feat: add Anthropic support #288

feat: add Anthropic support #288

Conversation

ocss884 commented Sep 14, 2023 • edited by dandansamax Loading

Description

Motivation and Context

Types of changes

Implemented Tasks

Checklist

Obs01ete left a comment

Choose a reason for hiding this comment

Obs01ete Oct 8, 2023

Choose a reason for hiding this comment

ocss884 Oct 13, 2023 • edited Loading

Choose a reason for hiding this comment

dandansamax left a comment

Choose a reason for hiding this comment

ocss884 commented Jan 3, 2024

Wendong-Fan commented Mar 14, 2024

coderabbitai bot commented Mar 17, 2024 • edited Loading

Auto Review Skipped

Chat

CodeRabbit Commands (invoked as PR comments)

CodeRabbit Configration File (.coderabbit.yaml)

Documentation and Community

ocss884 commented Mar 17, 2024

Wendong-Fan left a comment

Choose a reason for hiding this comment

Wendong-Fan left a comment

Choose a reason for hiding this comment

ocss884 commented Apr 1, 2024

ocss884 commented Apr 17, 2024

Appointat commented Apr 21, 2024 • edited Loading

lightaime May 25, 2024

Choose a reason for hiding this comment

Wendong-Fan May 26, 2024

Choose a reason for hiding this comment

ocss884 commented Sep 14, 2023 •

edited by dandansamax

Loading

ocss884 Oct 13, 2023 •

edited

Loading

coderabbitai bot commented Mar 17, 2024 •

edited

Loading

CodeRabbit Configration File (`.coderabbit.yaml`)

Appointat commented Apr 21, 2024 •

edited

Loading