[.Net][Feature Request]: Propose Orchestrator for managing group chat/agentic workflow in a more flexible way

[LittleLittleCloud]

Introduction

GroupChat is an orchestrator + [agents]

GroupChat is essentially a group of agents plus an admin agent to orchestrate the conversation. In each round, admin agent selects the next speaker from the group, and send the conversation context to selected speaker to generate the next reply.

However, given the complexity of agentic workflow, it's very challenging to propose an orchestrator that can satisfy all requests. And usually developers need to customize the orchestrator strategy based on their application, like error handling, retrying, etc...

Therefore, it would be beneficial to pull out the orchestrator into an individual component that can be easily customized. Following what @jackgerrits proposed in experimental branch, we can have an orchestrator interface defined like below

public class OrchestratorContext
{
    public IEnumerable<IMessage> ChatHistory { get; }
    
    public IEnumerable<IAgent> Candidates { get; }
}

public interface IOrchestrator
{
    // return null if no speaker is selected, this can be used to determine if the group chat need to be terminate or not.
    public Task<IAgent?> SelectNextAgentAsync(OrchestratorContext context, CancellationToken ct);
}

And we can provide the following built-in orchestrator

// the orchestrator used in dynamic group chat
public class RolePlayOrchestrator : IOrchestrator
{...}

// SequentialOrchestrator

// ...

And the CallAsync in IGroupChat can be simplified like below

for(i in max_round)
   var nextSpeaker = this.orchestrator.SelectNextSpeaker
   if (nextSpeaker == null) return; // terminate when nextSpeaker is null;
   var nextReply = nextSpeaker.GenerateReplyAsync(...)
   UpdateContext(nextReply)

Backgroud

Is your feature request related to a problem? Please describe.

There are a few issues about getting Sequence contains no matching element error when running a dynamic group chat.

• [Issue]: Write samle base on SemanticKernel with content copy from Example04_Dynamic_GroupChat_Coding_Task, can't work #2688
• [.Net][Bug]: GroupChat #2624

Why Sequence contains no matching element error happen in group chat

This is more like a feature rather than a bug in the current design. Because group chat relies on an admin agent to use LLM to dynamically decide the next speaker based on conversation. When deciding next speaker, admin uses prompt to require the LLM to respond in a specific format: From XXX where XXX is the name of the next speaker so the admin can extract the name correctly. However, there is no gaurantee that 1) LLM always return in the format of From XXX and 2) even LLM return XXX, XXX is one of the group members. When LLM fails to generate in correct format or hallucinate XXX that doesn't exist, the Sequence contains no matching element error will be thrown.

Why throwing the exception and break the conversation instead of using an alternative, more gentle strategy (like return a random agent) so the conversation can still be carried on

Throwing an exception is by design. Because without knowing of specific scenario, it's hardly a good choice to keep the conversation going which almost always makes sense only if we make some assumption. For example, in chatbot scenario, admin can fall back to select user as next speaker if LLM fails to generate the next agent. However, such strategy doesn't apply very well when in an agentic-workflow with retry policy.

How to configure the group chat admin to fall-back to default speaker

In the current group chat, you can use middleware over admin to customize the behavior of fall back strategy, the following example shows how to fall back to user for instance.

admin
   .RegisterMiddleware(async (messages, options, agent, ct) =>{
      var reply = await agent.GenerateResponse(messages, options, ct);
      var content = reply.GetContent();
      if (content is string text && (text == "from XXX") || ..... )
      {
         // the next speaker is among the group, return the reply
         return reply; 
      }

      // otherwise, always fall back to user as next speaker
      return new TextMessage(Role.Assistant, "from User", from: agent.Name);
   });

Additional context

No response