Support for LLaMa and Alpaca

[Chickensoupwithrice]

This project is awesome and I'm so glad it exists.

However I'm wonder about support dalai or text-generation-web-ui for running LLaMA and Alpaca. These models are much more suitable to be run on home computers and I believe would make an excellent addition to nixified.ai :)

I'd be open to helping as well, though my Nix-fu is not quite as powerful as yours.

[MatthewCroughan]

In case you're interested, it looks like llama.cpp already has a flake.nix at its root https://github.com/ggerganov/llama.cpp/

[Chickensoupwithrice]

This space moves quick. llama.cpp's nix flake works well, but does not provide an overlay, which is not necessarily a problem.
However there's new kids out on the block now too: Dolly and OpenAsisstant. Though both of these are largely focused on providing open datasets, building these projects with Nix is still a useful undertaking imo

[MatthewCroughan]

I was also thinking we could make nix functions which yield derivations that compute the prompt. Imagine

generateText "prompt" which would run llama.cpp in the sandbox. Relaxed Sandbox mode would allow access to the GPU.

[max-privatevoid]

Resolved via https://github.com/nixified-ai/flake/releases/tag/2. text-generation-webui is now available for Nvidia users.