Version 5 - Running Locally Again

Previously I was using models supplied by HuggingFace, specifically the google flan ones, and hosting the mentor machine on a HuggingFaceSpace. The issues were twofold; HuggingFaceSpace had resource limits, which especially hampered available models I could use on the free versions. The ones I could use could give one or two word answers at maximum, and many other times gave completely irrelevant or incorrect answers, due to how small (and old) they were. Ollama's models at this point seemed a better fit, as they were generally more powerful (and could actually code). I simply redownloaded the scripts and faiss index from huggingFace back onto my files, downloaded Ollama, then specifically its Mistral model, which I then switched in to replace the google flan models. The results were obvious and instantaneous. Compare the video here with the one from the previous version (4.0). The drawback is this version only works on LAN, and takes a ridiculously long time to download on account of Mistral, but it works very well. It takes roughly 200 seconds to run a decent prompt "How to make a Kraken pivot" as an example, which is quite long, but may be shortened in future versions. https://drive.google.com/file/d/1s8YeAYZP1uz_STmbUMtgFmOjYMhqF4SN/view?usp=sharing