Models still use outside knowledge in vault QA mode

[TroySigX]

• Disable all other plugins besides Copilot (required)
• Screenshot of note + Copilot chat pane + dev console added (required)

Copilot version: 2.8.2

(Bug report without the above will be closed)

Describe how to reproduce
Model: Qwen2.5:3b

To reproduce: Create 1 file (and the only file) that mentions a keyword (in my case, "confirmation bias) in a vault containing hundreds of other notes. Query the model to give the definition of that keyword (in my case, "what is confirmation bias")

Expected behavior
Model's source list should include that 1 file, and not use outside knowledge.

Screenshots

[logancyang]

Not every chat model is good for following instructions, I recommend some stronger chat models. It's not to say smaller models can't do well on RAG, it's just that they have to be particularly tuned for it.

This is a question and not a bug. Better for the discussion tab.

[TroySigX]

I tried to switch to llama3.2:3b-instruct-q8_0 and qwen2.5:3b-instruct-q8_0, but things aren't better, do you have any suggestion for which model I should use?

[logancyang]

3b is too small unless they are specially tuned for RAG. Try something larger than 32b that is distilled from a even larger model at least, ideally 70b+.

[motion2082]

32B would be pretty resource intensive on your PC. What 32B local models are you using?