Skip to content
/ node-llama-cpp Public

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

node-llama-cpp.withcat.ai

License

MIT license
1.5k stars 130 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

withcatai/node-llama-cpp

Error
Looks like something went wrong!

About

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

node-llama-cpp.withcat.ai

Topics

nodejs cmake ai metal json-schema gpu vulkan grammar cuda self-hosted bindings llama embedding cmake-js prebuilt-binaries llm llama-cpp catai function-calling gguf

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

1.5k stars

Watchers

17 watching

Forks

130 forks
Report repository

Releases 111

v3.8.1 Latest
May 19, 2025
+ 110 releases

Sponsor this project

 
Learn more about GitHub Sponsors

Used by 2.5k

  • @TNEM22
  • @sswiftdrtv
  • @elizaos-plugins
  • @martianina
  • @peterondesign
  • @taprwhiz
  • @venky24K
  • @ari-glasser
+ 2,459

Contributors 7

Languages