Performance issues under load testing (using teams_sso sample)

[maaisde]

Hello,

I am currently migrating our existing solution from the Microsoft Bot Framework to the new Agent 365 SDK for python. As a reference, I am using the teams_sso sample from this repository.

After completing the migration, we successfully passed all functional tests and were preparing for production deployment. However, we encountered significant performance issues during load testing.

Test Setup:

• I deployed the teams_sso sample locally and connected it to an Azure Bot Service resource.
• For load testing, I used the BotServiceStressToolkit. (https://github.com/damadei/BotServiceStressToolkit)
• The test involved 30 concurrent users.
• Each user triggers two types of requests per interaction: ConversationUpdate and Message.
• I am not using the SSO functionality for these tests—just simple requests that should immediately respond with:
• "Please type 'login' to sign in, 'logout' to sign out, or 'getUserProfile' to get user info."

Results:

• The average response time for ConversationUpdate requests is 23 seconds.
• The average response time for Message requests is 11 seconds.
• These response times are far too high for production use.

Questions:

1. What could be causing such long processing times when using the Agent 365 SDK, even with the basic sample and without SSO?
2. Are there known limitations or configuration steps required to improve performance under load?
3. Are there best practices or recommendations for optimizing the Agent 365 SDK for high concurrency scenarios?
4. Is there any guidance on how to properly scale or configure the SDK and/or Azure Bot Service for better throughput?

Any insights or suggestions would be greatly appreciated, as we would like to proceed with our migration but need to ensure the system is production-ready.

Thank you very much for your support!
Best regards,
Mario

[rodrigobr-msft]

Thank you, Mario. I will investigate this and get back to you within a day or two.

[rodrigobr-msft]

Also, which version of the SDK are you running?

[maaisde]

Also, which version of the SDK are you running?

Hi @rodrigobr-msft. My app is running on agent sdk version 0.3.2

[rodrigobr-msft]

Apologies, but this will take a bit longer to resolve than anticipated.

[maaisde]

Thank you for the honest update — I really appreciate it. No worries on our side, as we fortunately caught the issue before going live, so it’s not time-critical for us.

[rodrigobr-msft]

Do you know how the performance of Agents 365 SDK compares to Bot Framework? Is Bot Framework significantly faster?

[maaisde]

Same test setup. Only with the bot-conversation-quickstart sample and with a simple modification to get a quick response.
"Processing your message..."

https://github.com/OfficeDev/Microsoft-Teams-Samples/tree/main/samples/bot-conversation-quickstart/python

class BotActivityHandler(TeamsActivityHandler):
    def __init__(self):
        super().__init__()

        # Registers an activity event handler for the message activity event.
        # This is triggered whenever the bot receives a message.
        self.on_message_activity = self.handle_message

    async def handle_message(self, turn_context: TurnContext):
        """
        Handles incoming messages sent to the bot.

        Args:
            turn_context (TurnContext): Provides context for the current turn of conversation.
        """
        await turn_context.send_activity("Processing your message...")
        return

Results:

• The average response time for ConversationUpdate requests is about 3 seconds.
• The average response time for Message requests is just under 4 seconds.