bug: RAG follow-up questions show lost in the middle issue

[Propheticus]

Describe the bug
After attaching even a small 8 page document; When asking a follow-up questions about a topic that is discussed in the middle, no correct answers are given.
When you ask a question about this topic in the first question of a thread, the info is found and the answer is correct.

Steps to reproduce
Steps to reproduce the behaviour:

1. Attach a document in the first chat of a thread and ask for a summary
2. In the second prompt ask a follow-up question about a topic discussed midway the doc.
3. See the response stating nothing is said about that topic in the provided context. (false)

Expected behaviour
Using a model like Mistral 7B v0.2 I expect no lost in the middle issue, even when using a larger context.
In comparison: when I use AnythingLLM for embedding/vectorDB, either with or without pinning (pasting of the entire doc in context), the questions áre properly answered.

Screenshots
If applicable, add screenshots to help explain your issue.
Second question, nothing found:

First question, found without issue:

Second question via AnythingLLM, found without issue:

Environment details

• Operating System: Win 11 Pro N x64
• Jan Version: 0.4.9-345
• Processor: Ryzen 7 7700X
• RAM: 32GB
• GPU: AMD RX 6800XT 16GB
• Model: Mistral instruct 7B v0.2 Q5_K_M, max response tokens: 4096, Context length: 30K tokens.
• Retrieval settings:
  

Logs
No errors. They state a successful response was generated....which it was, just the content is of low quality.

[haliliceylan]

+1, Same problem here.

[louis-jan]

We will rework on RAG with separate embedding models.

[0xSage]

dupe of janhq/cortex.cpp#1595