Node.js OpenVINO LLM C++ addon

This is a Node.js addon for OpenVINO GenAI LLM. Tested using TinyLLama chat 1.1 OpenVINO int4 model on Windows 11.

Build

Run the following commands to build:

npm install
node-gyp configue
node-gyp build

To test the Node.js OpenVINO LLM addon run the index.js script.

node index.js D:/demo/TinyLlama-1.1B-Chat-v1.0-openvino-int4

Disable streaming

node index.js D:/demo/TinyLlama-1.1B-Chat-v1.0-openvino-int4 nostream

Supported models are here

Name		Name	Last commit message	Last commit date
Latest commit History 11 Commits
include/openvino		include/openvino
lib/intel64/Release		lib/intel64/Release
.gitignore		.gitignore
binding.gyp		binding.gyp
index.js		index.js
ovllm.cpp		ovllm.cpp
package-lock.json		package-lock.json
package.json		package.json
readme.md		readme.md