Repository navigation
Chopper: a small open model that cuts only the needed sentences out of retrieved text #8477
wiiiktor
started this conversation in
Show and tell
Replies: 1 comment 1 reply
|
wiiiktor Wiiiktor, preserving chopped sentences word-for-word to guarantee traceability is a brilliant architectural choice that directly mitigates hallucination in RAG pipelines. The 10x token reduction isn't just cost-saving; it forces the larger LLM to attend strictly to relevant context, which explains the jump in complex reasoning accuracy without introducing new noise. Keeping the source text intact ensures the model can't hallucinate facts it never saw. |
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi! Sharing a small open-source tool that may be useful here.
https://github.com/wiiiktor/chopper_demo
A retriever brings back 20 text snippets, but usually only one or two sentences in them answer the question. The chopper is a small model (0.3B parameters, runs locally) that cuts out only those sentences, word for word ("chops"). Nothing is rewritten, so every chop can be traced back to its source. If the answer is not there at all, it says so, so you can search again.
Test made with Qwen3-14B answering 400 questions, one retrieval each:
pip install chopper-ragworks with LangChain, LlamaIndex, Haystack or plain Python. Models (MIT): https://huggingface.co/wiiiktor/chopper_mmbertFeedback is very welcome, especially where it fails.
All reactions