A high-performance, local AI character integration using Node.js (ES Modules) and Ollama. This project is specifically tuned for the Intel i5-1235U with 16GB RAM, focusing on bypassing Windows DNS latency and providing real-time streaming.
- Custom Personality: A "War-Machine" persona defined via a dedicated
Modelfile. - Zero-Lag Networking: Direct IPv4 (
127.0.0.1) binding to skip the 2-second Windowslocalhostlookup delay. - Real-Time Streaming: Uses HTTP chunked encoding to deliver words the millisecond they are generated.
- RAM Optimized: Includes
keep_alivelogic to ensure the model stays in your 16GB RAM.
- Processor: 12th Gen Intel Core i5-1235U (2 P-Cores, 8 E-Cores).
- RAM: 16GB DDR4/DDR5.
- Performance Note: On this CPU, expect roughly 5-8 tokens per second. Streaming is enabled to ensure "First Token" delivery in < 1.5s.
Download the engine at ollama.com. Ensure the Ollama icon is visible in your system tray.
# Initialize and install dependencies
npm init -y
npm install express ollama
npm pkg set type="module"Ensure you have a file named Modelfile in your root directory. Then, register the character:
# Run via terminal
ollama create war-machine -f Modelfilenode server.jsThe primary endpoint for interacting with War-Machine.
Headers: Content-Type: application/json
Request Body:
{
"prompt": "War-Machine, what is your current status?"
}Testing via PowerShell (cURL):
curl.exe -X POST [http://127.0.0.1:3000/ask](http://127.0.0.1:3000/ask) `
-H "Content-Type: application/json" `
-d '{"prompt": "Give me a status report on the CPU cores."}'- Direct IP: Changed
localhostto127.0.0.1in the client to stop DNS lag. - Streaming Loop: Implemented
for await (const part of stream)to pipe output directly. - Keep-Alive: Added
keep_alive: '30m'to prevent the i5 from reloading from SSD.
Below is a live look at War-Machine in action. Notice the low-latency streaming and the character-driven persona.
Note: On an i5-1235U, the first token is delivered in ~1.2s, with a full response completed in under 9s thanks to streaming optimizations.
MIT - Created for the War-Machine Project. 🤖🦾
