[Performance] Latency is big issue with ms bot framework

[venkey1000]

Hi,
I have developed one test bot using ms bot framework, there is so much latency in replies from the bot.
When ever user says hi, my bot also greets user, its just a simple text message preparation from app side to send it back to the user, even this whole process is taking around 10 sec with out any back end api calls to any service. I don't know where the issue is. Can please clarify on this latency issues.

[palmerabollo]

@venkey1000 Take a look at this issue that might clarify some things #915 (10 seconds is too much in any case). The issue #1395 is also related.

I agree that latency in the directline channel has a lot of room for improvement.

[jameslew]

Using Application Insights to track calls can be really valuable from a diagnosis perspective; and at development levels of traffic it's free to use (up to millions of events currently).

There shouldn't be any reason a request would take 10s; useful information for us would be what region your bot is deployed in, what region your user is in, and which channel you're using.

[venkey1000]

Hi @palmerabollo, @jameslew , my bot is deployed in India, if i move the deployment of bot to US, will it make any difference in latency? I am using facebook channel. Here are some of the application insights that i have found in the case of greeting user with hello message, i am accessing session stored values to get the name of the user.

Time taken(in seconds)
user->app start point (1 sec)
luis dialog match (1 to 3 sec)
after luis dialog match to preparing prompt inside dialog(1 to 2 sec)
prompts.send to user(till he receives the message) (3 to 4 sec)

Please let me know if if you find any thing from insights this case to improve latency.

[eyalsh]

I'm seeing a similar issue. There's a significant several seconds delay from the moment I'm sending a message in the bot code until it get's delivered to the FB Messenger client.
I'm developing on Linux using ngrok and I'm based in Israel.

When I deploy the bot to Azure on West-US the delay is gone and response time is good.
However, since I'm using the local setup + ngrok for development this huge delay is a major issue.

[somprabhsharma]

@jameslew I am facing similar issue for telegram, slack, kik, messenger, skype. There is significant delay in the response. I tried creating a bot which directly interacts with the channel (without using MBF) then the response is pretty fast. But if I use MBF then there is at least 5 second delay in the response.

My bot is deployed on Amazon in us-east-1 region.

[tomlm]

Up until about a week ago we were only in West and East US data centers. We now have BF data centers in SEAsia and North Europe, which should help quite a bit. That said, for best results you should consider having your bot deployed into multiple data centers as well to have it be as local to the BF and channel services as possible.

[astrung]

"That said, for best results you should consider having your bot deployed into multiple data centers as well to have it be as local to the BF and channel services as possible" >> How can i do it ?

[spreddy2714]

I am also facing this huge latency issue.
Isn't there any other way that moving the bot to US region?

[jameslew]

@spreddy2714, tell us more about where your bot is deployed and what service you're using. Its also useful to know if you're using the bot state store, luis, etc as these all take round trips to the cloud.

[spreddy2714]

Bot is deployed in India. I am predominantly using on luis service & rest api calls. this is the core for my bot. Please suggest me if there are any possible tweaks/options to make it more faster.
I've used it with trail version on azure, it was fast and instant. Then I moved it to my production server(local private cloud, where the whole communication backend rest api is available.)
Then the results got too slow, but local emulator is still quick(don't understand how?).

[astrung]

So should we use Azure server for bot ? i think we need a document for it on Bot Documents

[jerinkurian]

I am also facing the same issue the bot is getting too much time to respond. First response is taking around 10 - 12 seconds then onwards 5-6 seconds. I am developing this bot for prestigious customer and the customer is now taking a step backward to release the bot due to this performance issue. Can anybody help?

[jameslew]

@spreddy2714 you're definitely paying a cross-world latency tax then for your LUIS calls, as that dc is in the US. We're expanding our presence world-wide soon so there should be a datacenter closer to you.

That said, do you have any sort of trace representing the latencies you're seeing? Are you seeing the Bot Builder and seeing bot state store requests as well?

[jameslew]

@astrung there is no reason you need to use Azure servers for bot, though of course they work really well and we'd be happy to have you use them :). Wherever you deploy though, if there's a traffic management solution (such as Azure Traffic Manager and I'm sure AWS has something as well), they can help with routing. That said, if your users are in a similar region to where your bot and the services you depend on are, there's no need for such a distribution.

[jameslew]

@jerinkurian Depending on how you've built your bot, that first request can be warm up time of the service your bot is hosted on; ie a cold bot on azure servers can take a few moments to warm up that isn't seen on subsequent requests.

Do you have any more information on where the time is being spent in your bot? Have you done any profiling of which calls are taking a long time?

[jerinkurian]

@jameslew

Thank you for your reply :)

Yes the first call is always taking some time, will it be possible to narrow down this response time?

Here is a brief explanation what I am doing with my bots

Bot #1
In this bot I have a DB call(MS SQL DB hosted in azure) and few checking to the state service. I found that is taking atleast 2- 3 seconds to reach the message to the bot application. Then after DB checking and state service checking it is taking another 3 seconds so total 6 seconds.
Another problem I found that the bot is not at all responding even after 6 seconds. Feels like it is freezing and start responding after sometime. For example I asked to bot "hi" then no reply again "hi" still no reply but after sometimes I get the answer for both "hi" at the same time.

Bot #2
This is just a reply bot, whatever you are asking it will give you an echo. There is no DB check and state service check. Still it is taking 3 seconds to reply.

Some of my thoughts
My bot is deployed in Azure, and I am using the basic web app. If I use a standard price tier web app in azure to deploy the Bot will it make any difference? Same applies to MS SQL DB.

[nwhitmont]

@jerinkurian @venkey1000 are you using C# or Node SDK for BotBuilder?

[venkaiahj]

@nwhitmont I am using node js SDK

[nwhitmont]

A lot of things are changing in Bot Framework this week. If you are still experiencing the problem, open a new issue. Thx

[ivikashjha]

My Bot run perfectly fine in local machine, but i deploy in azure and test in azure, it behaves weird. Waterfall steps are sometimes not correct, it start from first step anywhere from the middle with correct input as well.

[EricDahlvang]

Hi @ivikashjha

Issue 1413 is closed, and the team does not monitor closed issues. The preference is for usage and howto questions be directed to Stack Overflow. Please ask this question on https://stackoverflow.com/questions/tagged/botframework and provide the startup code and the waterfall dialog that is having the issue. Our support team monitors Stack Overflow daily, and will assist you through to resolution.

[ivikashjha]

Stack Over Flow: Sorry, we are no longer accepting questions from this account.