Repository navigation
C++ Wrapper with Decisions #30765
MartinPerry
started this conversation in
Show and tell
C++ Wrapper with Decisions
#30765
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I have created a simple C++ wrapper on top of the Llama API. It does not have all the functionality that API offers. I have added only what I needed. I wanted to use just the pure C API - not tools (except for mtmd, which has a C API as well) and not the common library.
Chat wrappers currently do not support Jinja templates; I have a callback and process templates manually.
Lately, I needed decisions directly in C++, so with the use of Codex, I have reimplemented
tools/server/server-decision.cppand the tools that it uses as a bunch of separate classes. The code is implemented against llama 0.7.0.Repo: https://github.com/MartinPerry/LLamaCppWrapper
It's still WIP based on what I currently need :-) However, I wanted to share as someone may find it helpful. Note that there are probably bugs, since I have not tested all models and all variants. The code is also not the best out there, but it's OK for my experiments and easy to read/modify.
All reactions