Repository navigation
Keeping a private, offline model up to date without retraining #2460
troybrandonc-bit
started this conversation in
Ideas
Replies: 1 comment
|
One thing I should add, to be fair: the 26.7% LoRA number above is plain LoRA, without replay. If you re-train LoRA on everything from before each night, it keeps facts about as well as we do, I tested that too. Where Fintan still wins is learning new skills (95.9% vs 72.2%, and 89.2% vs 64.0% on a topic it hadn't seen), keeping the rest of the model exactly as it was, and cost, since replay gets bigger and slower every night and we don't re-train on the past. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi Zylon team,
I'm Troy, I run a small lab in Spain. You run open models inside banks and other places that can't send data out, so I'd guess one pain is keeping the model current: rules, products and procedures change, and retraining or swapping the model is slow.
We build Fintan, a learning layer that sits inside the model you already run. Every night it learns new documents into the model's own weights, fully offline. Nothing about your setup changes: same model, same serving (it works with vLLM).
Results on open models:
I'd like to try it on public documents like your customers' own, for example a regulator's rulebook and its updates. Would that be useful to you?
Troy
All reactions