Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit

* feat: export `llama.cpp` release info
* feat: make it easier to use imported `build` and `download` commands from other libraries

Git stats


Failed to load latest commit information.
Latest commit message
Commit time
October 25, 2023 22:45
August 13, 2023 00:03
October 25, 2023 22:45
October 25, 2023 22:45
November 6, 2023 20:12
October 25, 2023 22:45
October 25, 2023 22:45
August 13, 2023 00:03
August 13, 2023 00:03
August 13, 2023 00:03
August 13, 2023 00:03
August 29, 2023 01:32
October 25, 2023 22:45
node-llama-cpp Logo


Run AI models locally on your machine

Pre-built bindings are provided with a fallback to building from source with cmake

Build License License Version


  • Run a text generation model locally on your machine
  • Metal and CUDA support
  • Pre-built binaries are provided, with a fallback to building from source without node-gyp or Python
  • Chat with a model using a chat wrapper
  • Use the CLI to chat with a model without writing any code
  • Up-to-date with the latest version of llama.cpp. Download and compile the latest release with a single CLI command.
  • Force a model to generate output in a parseable format, like JSON, or even force it to follow a specific JSON schema



npm install --save node-llama-cpp

This package comes with pre-built binaries for macOS, Linux and Windows.

If binaries are not available for your platform, it'll fallback to download the latest version of llama.cpp and build it from source with cmake. To disable this behavior set the environment variable NODE_LLAMA_CPP_SKIP_DOWNLOAD to true.


import {fileURLToPath} from "url";
import path from "path";
import {LlamaModel, LlamaContext, LlamaChatSession} from "node-llama-cpp";

const __dirname = path.dirname(fileURLToPath(import.meta.url));

const model = new LlamaModel({
    modelPath: path.join(__dirname, "models", "codellama-13b.Q3_K_M.gguf")
const context = new LlamaContext({model});
const session = new LlamaChatSession({context});

const q1 = "Hi there, how are you?";
console.log("User: " + q1);

const a1 = await session.prompt(q1);
console.log("AI: " + a1);

const q2 = "Summerize what you said";
console.log("User: " + q2);

const a2 = await session.prompt(q2);
console.log("AI: " + a2);

For more examples, see the getting started guide


To contribute to node-llama-cpp read the contribution guide.


Star please

If you like this repo, star it ✨