Use this as an api from another python file?

[regstuff]

Hi,
First off, thanks for the OPENBLAS tip. That cuts down the initial prompt processing time by like 3-4x!
Was wondering if its possible to use the generate function as an api from another python file.
Secondly, is it possible to update to the latest llama.cpp with a git pull from the llama.cpp library or do I have to wait for you to sync changes and then git pull from koboldcpp.

[LostRuins]

Yes, the API is a KoboldAI compatible API and can be used by any external program. Take a look at the attached file, the main endpoint that you want to use is /api/v1/generate

The repo is manually synced with the parent one every few days, but you'd probably want to wait for me to pull updates manually as there are often breaking incompatibilities that arise after merging which I have to fix before making a new version. It's not exactly plug and play due to the multiple changes I've made.

koboldapi-documentation.pdf