Welcome message on_chat_start not appearing in UI

[JamesLloydPartingtonSerendipityai]

Describe the bug
When using Safari (even on private window), when sending a message in the on_chat_start callback the message frequently does not appear (over 90% of the time). Works fine in Chrome. Messages sent in the on_message work fine too.

Issue started occurring when we updated from Chainlit 1.0.301 to Chainlit 1.0.502, downgrading fixed the issue.

This issue does not happen when running locally, but does happen when deployed on Kubernetes.

Desktop:

• Apple M2 Pro
• Version 13.5, Ventura
• Safari Version 16.6

[constantinidan]

Hi @JamesLloydPartingtonSerendipityai can you share a screenshot of the error message?

[JamesLloydPartingtonSerendipityai]

Hi @constantinidan no errors appear on the UI, console or backend. Also noticed the settings do not appear when this happens. We configure the settings in on_chat_start.

[tpatel]

Hi, it might be linked to an Azure issue: microsoft/azure-container-apps#812.

[JamesLloydPartingtonSerendipityai]

Hi @tpatel , If that was the issue, then downgrading to a different version of Chainlit would not have fixed the issue, also why would this only occur in Safari? (Using AWS btw)
Looking more at the sockets though I do notice one thing. When running successfully, the Switching Protocols occurs only once. However, it always happens twice when the message and settings do not appear.

[jimmarshall87]

Hi guys, just to report that @JamesLloydPartingtonSerendipityai and I tracked this down and managed to workaround it, so I thought you would like to hear the details @tpatel @willydouhard @constantinidan.

First, the facts:

• We found we had this issue in Safari on Mac or iPhone but not on Chrome. Safari on Mac we updated to the latest version with no improvement.
• This issue did not occur locally, only when we are deployed on our two pod AWS setup
• We went through version by version and found that the issue was introduced in chainlit 1.0.500 and was not present in prior versions. The issue persists to the latest version of chainlit.
• The behaviour you see is that sometimes your initial opening message in Chainlit is not displayed as James describes above.
• We also occasionally saw a console message that looks like this:

WebSocket connection to 'wss://<our url>/ws/socket.io/?EIO=4&transport=websocket&sid=-P8qtONaTloNTEknAABY' failed: WebSocket is closed before the connection is established.

Next, we went through all the commits in that chainlit release and unfortunately it was quite a big one. We identified a few candidates and then tried a few hypothesis.

We noticed you did something with gzipmiddleware in that release and so we tried disabling it, just after importing chainlit in the startup sequence, like this:

for i, middleare in enumerate(app.user_middleware):
    if isinstance(middleare.cls, GZipMiddleware) or middleare.cls == GZipMiddleware:
        app.user_middleware.pop(i)

This immediately made it work, on both Mac Safari and iPhone Safari. We don't really understand the details here, but maybe this gives you some clues as to what is going on here - or may help others in the same situation!

Wondering if this could be added as an optional configuration to allow us to take this hack out! Does the above help shed any light on why this might be happening?

[constantinidan]

Thanks a lot @jimmarshall87 @JamesLloydPartingtonSerendipityai

We'll take a look today or tomorrow!