Option to stream response token by token

[ranfdev]

No description provided.

[zhudotexe]

Although streaming wasn't in our original design since we felt like it was unnecessary for most academic and many production-system use cases, I've been thinking about methods to implement it with the following constraints:

• No breaking changes - existing 0.x code should not need to account for a new return type, for example
• Minimal added complexity to the Engine interface, optimally it should be an optional implementation s.t. streaming code can still interchange with non-streaming engines
• Compatibility with function calling - this might end up piggybacking off the optional constraint above

In my mind the interface would look something like:

async for msg_stream in kani.full_round_stream(...):
	# design consideration: what message stream would function calls emit?
	# msg_stream should be a StreamManager object so that you can access header data like role before consuming
	async for token in msg_stream:
		# stream code here

	# to get the full content as a ChatMessage - this would consume the full stream if called before finished
	msg = await msg_stream.message()

I don't think this'll be implemented in 0.x but I'd like to implement this as a 1.x feature.