LangChain LLM

[lukestanley]

Hi, I noticed there was interest in using LangChain with Alpaca and you did a lot of the work already needed for streaming so I wondered if I could built on it and make it into a LLM class, here is my progress so far, just on a Gist right now:
https://gist.github.com/lukestanley/6517823485f88a40a09979c1a19561ce
I mention it in this existing LangChain issue:
langchain-ai/langchain#1777
Obviously feel free to do what you like with my small contribution.

[plattenschieber]

Are you still considering this @lukestanley? Did you put something in production using serge internally?

[lukestanley]

LangChain's llama.cpp class now has good support for streaming. I extended the exiting class in the end and the code is live in LangChain Python and I'm happy with it. @plattenschieber