Enable OpenAI model streaming and fix num_tokens_from_messages

[zechengz]

Description

• Enable stream for OpenAI model and handle stream mode in ChatAgent
• Improve the translator example
• Fix num_tokens_from_messages

Following is an example for translator

python3 translation/translator.py --single --language chinese --stream

for a simple input json file

{
	"message_1": {"content": "My name is Zecheng and I am from China."},
	"message_2": {"content": "I love the CAMEL Python package and now I am a contributor to the package!"},
	"message_3": {"content": "This is a CAMEL translator example :)."},
	"num_messages": 3
}

Motivation and Context

Parts of #178

Some changes for future PRs:

• Support other agents with the stream mode and check whether stream mode works for role playing etc.
• Deprecate print in other place if possible.

Types of changes

What types of changes does your code introduce? Put an x in all the boxes that apply:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds core functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)
• Example (update in the folder of example)

Checklist

• I have read the CONTRIBUTION guide. (required)
• My change requires a change to the documentation. (minor change)
• I have updated the tests accordingly. (required for a bug fix or a new feature) (Will add unit tests after first round of reviews.)
• I have updated the documentation accordingly.

[Obs01ete]

A library can NOT use print (and input) as its core functionality. RolePlaying step should be split into single steps and the step should return an iterator that yields words or chunks as they are received as deltas. All printing should be done in user code: examples, apps etc. Even though CAMEL is currently using print and input we are going to move away from this to a better API. I suggest to not merge this.

[zechengz]

@Obs01ete Several questions,

• Are we going to support any terminal experience in the future with printing response directly? If not I will remove print directly. If yes, I may still leave it as an option for users. Let user specify whether print response directly in the terminal
• Now we return ChatAgentResponse with msgs as List[ChatMessage] for step(), if we are going to change the return value to iterator, I will move functions that handle model output (for batch and streaming) to some other place such as utils
  Wdyt? Thanks in advance.

[Obs01ete]

@zechengz thanks for your comments. We are not going to support terminal prints to display the answers starting from the nearest future. Most likely we will rework ChatMessage into a future object supporting .get() and a chunk generator for streaming mode.

[zechengz]

@Obs01ete I have updated the PR. For now I still parse the streaming output same as that batch output. Other modification please refer to the PR description. Please take another round of review if possible. Thanks in advance!

[lightaime]

Thanks @zechengz! Looks great overall. Left some comments.

@Obs01ete if we do not support any logging how stream is supposed to be usefull?

[lightaime]

What is the difference between this and num_tokens_from_messages?

[zechengz]

It seems that openai usage for completion tokens does not include tokens_per_message etc in the num_tokens_from_messages

[lightaime]

LGTM! Thanks @zechengz

[Obs01ete]

Thanks @zechengz! Looks great overall. Left some comments.

@Obs01ete if we do not support any logging how stream is supposed to be usefull?

The Agent API will have a streaming mode. This way an app like Agents can print the chat in real time as OpenAI API generates it.