Update speaker selector in GroupChat and update some notebooks

[thinkall]

Why are these changes needed?

There are a lot of ask for having more control on speaker selection. In this PR, several speaker selection methods are enabled. A parameter allow_repeat_speaker is also added to determine whether to allow the same speaker to speak consecutively.

    - speaker_selection_method: the method for selecting the next speaker. Default is "auto".
        - "auto": the next speaker is selected automatically by LLM.
        - "manual": the next speaker is selected manually by user input.
        - "random": the next speaker is selected randomly.
        - "round_robin": the next speaker is selected in a round robin fashion.
    - allow_same_speaker: whether to allow the same speaker to speak consecutively. Default is True.

groupchat_RAG notebook is updated accordingly.

Besides:

• this PR fixed some notebooks which were not updated accordingly when seed is renamed to cache_seed.
• RetrieveChat notebook is updated so that it can run by just click "Run All". No need to prepare reference docs for FLAML.

Related issue number

Closes #151

Checks

• I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[codecov-commenter]

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (f939dda) 29.05% compared to head (0e230b6) 38.02%.

Files	Patch %	Lines
autogen/agentchat/groupchat.py	91.66%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #688      +/-   ##
==========================================
+ Coverage   29.05%   38.02%   +8.96%     
==========================================
  Files          27       27              
  Lines        3407     3448      +41     
  Branches      767      780      +13     
==========================================
+ Hits          990     1311     +321     
+ Misses       2345     2018     -327     
- Partials       72      119      +47     

Flag	Coverage Δ
unittests	37.96% <91.66%> (+8.96%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[thinkall]

[Edit]: Found the bug. Please fix. (see my review)

====

I'm seeing some cases where the speaker is allowed to repeat, and I'm trying to sort out why.

Configuration:

groupchat = autogen.GroupChat(
        agents=[user_proxy, assistant, distraction_agent, guardrails_agent],
        messages=[],
        speaker_selection_method="auto",
        allow_repeat_speaker=False,
        max_round=15)

And the logs:

--------------------------------------------------------------------------------
guardrails_agent (to chat_manager):

You're welcome! I'm glad I could assist you. Feel free to reach out whenever you need help. Have a wonderful day too!

--------------------------------------------------------------------------------
guardrails_agent (to chat_manager):

The conversation is acceptable by the responsible AI policies. You can proceed with your discussion or inquiry. How may I assist you today?

--------------------------------------------------------------------------------

Thanks @afourney , I think it's OK to just remove _participant_roles from the prompt to fix the repeat speaker bug. Could you help confirm?

[afourney]

Thanks @afourney , I think it's OK to just remove _participant_roles from the prompt to fix the repeat speaker bug. Could you help confirm?

No, we cannot remove _participant_roles, as this is a major change, and a key way that the GroupChat knows what each agent is capable of (otherwise it's only guessing from the agent names).

The right solution here is to add an optional agents parameter to _participant_roles, (default it to self.agents). That way we can pass it the reduced list of agents.

This also needs to be fixed in a similar way:

        # Return the result
        try:
            return self.agent_by_name(name)
        except ValueError:
            logger.warning(
                f"GroupChat select_speaker failed to resolve the next speaker's name. Speaker selection will default to the next speaker in the list. This is because the speaker selection OAI call returned:\n{name}"
            )

[thinkall]

Thanks @afourney , I think it's OK to just remove _participant_roles from the prompt to fix the repeat speaker bug. Could you help confirm?

No, we cannot remove _participant_roles, as this is a major change, and a key way that the GroupChat knows what each agent is capable of (otherwise it's only guessing from the agent names).

The right solution here is to add an optional agents parameter to _participant_roles, (default it to self.agents). That way we can pass it the reduced list of agents.

This also needs to be fixed in a similar way:

        # Return the result
        try:
            return self.agent_by_name(name)
        except ValueError:
            logger.warning(
                f"GroupChat select_speaker failed to resolve the next speaker's name. Speaker selection will default to the next speaker in the list. This is because the speaker selection OAI call returned:\n{name}"
            )

Oh, yes, I missed the agent.system_message. Please feel free to edit the code.

[afourney]

I believe I have fixed the bugs that were found earlier I am hardly an unbiased reviewer now, however.