[BUG] Conversational Retrieval QA Chain behaves different on flowise and embed website

[emrahtoy]

Describe the bug
The chain behaves and replies differently on both sides.

To Reproduce
Create Conversational Retrieval QA Chain chat flow based on the template or created yourself. I used a text file document with an in-memory vector store.

1. Limit your prompt within the border of the document or use the default prompt which works same way. I also added my own prompt.
2. Test your chat flow on Flowise editor chat panel.
3. Say "Merhaba" or "Hello" ( merhaba means hello in turkish language )
4. Do the same thing on your website with an embedded Flowise chat panel.
5. Also forgets the language I limited. ( you can use the document and prompt I added below )

Expected behavior
Both should give the same answers regarding to the document given
Flowise editor chat panel works as expected and relies on variables i set on chat model.
The embedded website panel does not care about the content of the document and gives answers unrelated to the prompt and doesn't care about variables I set on the chat model. Also forgets about prompted language even though I strictly limit it.

Screenshots

Document ( 1500 chunk default variables on character text splitter )
YTU Startup House.txt

Prompt
I want you to act as an officer working at Yıldız Technical University. Your name is Ahmet. Stay within the boundaries of the document. Reply, answer, write, and talk in Turkish only. When something irrelevant is asked or said reply "Size Yıldız Teknik Üniversitesi Startup House hakkında yardımcı olmak için buradayım." shortly in Turkish and then redirect people to the boundaries of the document politely. Be helpful and do not give long answers. Never break character.

[emrahtoy]

Logs are giving kind of a hint about behavioral difference.

embed.log
flowsiePanel.log

[emrahtoy]

@HenryHengZJ how can I debug components?

[emrahtoy]

Logs show that the Embed version is jumping the "[1:chain:ConversationalRetrievalQAChain > 2:chain:StuffDocumentsChain]" step.

To me, it should have got both the text ( in base64 format ) and the chain data from chat flow data requested from the database.

[HenryHengZJ]

@HenryHengZJ how can I debug components?

you can put console log in the component file

I tried it there it seems to behave same for me:
Flowise Chat:

Embedded Chat:

[emrahtoy]

So strange, I tried again and have had different results. I will clean everything and try again.

[emrahtoy]

@HenryHengZJ it is still happening. Plus son that there are new bugs... embed gets 401 unauthorized error at first but then receives stream ( probably over websocket ) and API requests causes another error ( when you have a brand new flow and trigger it over API at the first time )

I still see the difference I mentioned before in the logs. I am trying really hard to use flowise with our products but keep facing with the new bugs. I would like to contribute more but I am seriously in need of help to learn how to debug this project end to end ( by debug I don't mean console.log.. I mean debug points where I can see call stack, etc.. )

[HenryHengZJ]

@emrahtoy do you have some projects in mind that have good debug points where we can take reference from?

when error happens, logs like this should be able to tell you where the error is from:

[emrahtoy]

Any kind of log only gives information about the error and maybe call-stack at limited deep. On the other hand, debugging with a debug point gives you the state of the app and information on the variables inside the current context.

For example, you have no idea what was the json string giving the error above. If you could set a debug point on line 43 of the PromptTemplate.js file then you could see the JSON string because the application flow would stop at that point right before the error happens. Most probably you are not going to able to do that because of the structure of the mono repo and how modules connected to each other in development mode.

You are using distribution packages of flowise-ui and flowise-components in the server module, this is where the "yarn link" comes in handy as it will make it to use the real dist folder inside the workspace. Yet still not enough. At this point, source maps may help as stated here but I couldn't spare enough time to test.

[mibtim]

Henry and emrahtoy, hope all is well. guys just checking into see if this issue embed issue is resolved? for it doesn't still work. thanks

[emrahtoy]

@mibtim i believe it is not resolved. I figure out how to debug chains and I will check again today

[mibtim]

thank you heaps mate, please keep me updated thanks heaps

[HenryHengZJ]

@emrahtoy @mibtim see update here - #473