pondok-ai

Run your own private AI stack locally with one command.

Local LLMs, a chat UI, vector search, and an OpenAI-compatible API, wired together with Docker Compose.

English · Bahasa Indonesia · 한국어 · العربية · Español · 中文 · Français · 日本語 · Deutsch · Português · Русский · हिन्दी · Türkçe · Italiano · Tiếng Việt · Polski · Bahasa Melayu

_{Translations are machine-generated and may lag behind the English README.}

_{From an empty folder to a reply from a local model. Recorded on an Apple M1 Mac with a native Ollama (Metal GPU), images and model pre-pulled; the wait for Open WebUI to start is sped up and labelled on screen. The Mac had less than 10 GB of free disk, so init runs with --force.
▶ Watch the 1080p video}

Quickstart · Features · Screenshots · Presets · Share on your network · CLI · API Gateway · Docs

pondok-ai is a CLI that boots a complete, private AI environment on your own machine. It generates a Docker Compose stack for your hardware, starts it with health checks, and manages models for you.

Prompts, chats, and embeddings are processed on your machine. Pulling container images and model weights the first time requires an internet connection.

npx pondok-ai init
npx pondok-ai up

_{Pondok is Indonesian for a small lodge or shelter. The package was formerly published as local-ai-stack.}

Prerequisites: Node.js 22 or later, Docker, and Docker Compose v2.

1. Initialize the stack

# Interactive setup wizard with hardware detection
npx pondok-ai init

# Or non-interactive with a chosen preset
npx pondok-ai init --preset standard --yes

2. Start the services (waits until every service is healthy)

npx pondok-ai up

The first start can take a few minutes while Open WebUI downloads its support models. On a Mac with native Ollama, up also starts Ollama if it is not running. If no model is installed yet, up tells you which one fits your RAM.

3. Pick and install a model

# Recommendations based on your RAM, VRAM, and free disk
npx pondok-ai model recommend

# Only models that answer well in Bahasa Indonesia (Gemma 3, Qwen 2.5, Aya Expanse, BGE-M3)
npx pondok-ai model recommend --lang id

# Install with a pre-flight memory and disk check
npx pondok-ai model install llama3.2:3b

Once running, open your stack:

curl -fsSL https://raw.githubusercontent.com/rasidi3112/pondok-ai/main/scripts/install.sh | bash

iwr -useb https://raw.githubusercontent.com/rasidi3112/pondok-ai/main/scripts/install.ps1 | iex

_{Open WebUI chatting with gemma3:1b, served by a native Ollama on an Apple M1 Mac.}

pondok-ai doctor checks Docker, RAM, disk, and GPU	pondok-ai model recommend sorts models by what your hardware can run

One machine can serve a whole classroom, office, or pesantren. Everyone on the same Wi-Fi or LAN opens Open WebUI in their browser, each with their own account and chat history, while the models run on that one machine.

_{The teacher runs pondok-ai share, a student opens the link on a phone and signs up, the teacher approves the account, and the student asks a question in Indonesian. Recorded on an Apple M1 Mac with gemma3:1b; the phone is a phone-sized browser on the same Wi-Fi, and nothing is sped up.
▶ Watch the 1080p video}

1. Start the stack, then open http://localhost:3000 on this machine and create the admin account.

   npx pondok-ai up

2. As the admin, open Admin Panel > Settings > General, turn on New Sign Ups and save. Open WebUI turns sign-ups off once the admin exists. Leave Default User Role at Pending.

3. Share Open WebUI on the local network:

   npx pondok-ai share

   It prints the address for each network this machine is on, such as http://192.168.1.20:3000, with a QR code people can scan with their phone. On a terminal with a light background, use npx pondok-ai share --light so the QR code scans.

4. People sign up from their own devices. Approve each one in Admin Panel > Users, and they can start chatting with the models installed on this machine.

It is safe by default:

• Admin first. share refuses to run until the Open WebUI admin account exists, because the first account to sign up becomes the admin.
• Login is always on. New accounts wait as pending until the admin approves them in Admin Panel > Users.
• Only Open WebUI is shared. The API gateway, Ollama, Qdrant, and n8n keep their bind address (127.0.0.1 by default), so nothing without a login is exposed.

While sharing, approved users can use every model installed on this machine, including ones you marked private in the Workspace. The "Account Activation Pending" screen shows the admin's name and email; turn off Show Admin Details in Account Pending Overlay in Admin Panel > Settings > General to hide them.

Traffic is plain HTTP, so share only on a network you trust, not public Wi-Fi. Your OS firewall may ask to allow incoming connections the first time. Stop sharing with npx pondok-ai share --stop.

┌───────────────────────────────────────────────────────────────┐
│                      Host Machine (CLI)                       │
│                                                               │
│   Port 3000                 Port 8000           Port 5678     │
│       │                         │                   │         │
└───────┼─────────────────────────┼───────────────────┼─────────┘
        ▼                         ▼                   ▼
  ┌───────────┐             ┌───────────┐       ┌───────────┐
  │ Open      │             │ API       │       │ n8n       │
  │ WebUI     │             │ Gateway   │       │ (Full)    │
  └─────┬─────┘             └─────┬─────┘       └─────┬─────┘
        │                         │                   │
        ├─────────────────────────┼───────────────────┤
        ▼                         ▼                   ▼
  ┌───────────┐             ┌───────────┐       ┌───────────┐
  │ Ollama    │             │ Qdrant    │       │ Postgres  │
  │ (LLMs)    │             │ (Vector)  │       │ (State)   │
  └───────────┘             └───────────┘       └───────────┘

See the Architecture Overview for details.

Images are pinned to explicit versions. pondok-ai update refreshes the configured versions; it does not select newer releases. After installing a newer CLI, run pondok-ai update --upgrade-images to migrate only the CLI's legacy default image tags. Custom image values are preserved.

The API Gateway provides drop-in OpenAI REST endpoints. Set PONDOK_AI_API_KEY to the generated API_SECRET_KEY before calling a /v1/* endpoint.

The gateway limits authenticated /v1/* requests per client (source IP and API key) to 120 requests per minute by default. Configure RATE_LIMIT_MAX_REQUESTS and RATE_LIMIT_WINDOW_MS in .env if your local client needs different limits.

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $PONDOK_AI_API_KEY" \
  -d '{
    "model": "llama3.2:3b",
    "messages": [
      { "role": "user", "content": "Hello from pondok-ai!" }
    ]
  }'

import OpenAI from 'openai';

const apiKey = process.env.PONDOK_AI_API_KEY;
if (!apiKey) {
  throw new Error('Set PONDOK_AI_API_KEY to your generated API_SECRET_KEY.');
}

const openai = new OpenAI({
  baseURL: 'http://localhost:8000/v1',
  apiKey,
});

const completion = await openai.chat.completions.create({
  model: 'llama3.2:3b',
  messages: [{ role: 'user', content: 'Explain quantum computing in one sentence.' }],
});

console.log(completion.choices[0].message.content);

See the API Gateway Reference for every endpoint.

• examples/openai-compat/: Drop-in OpenAI SDK integration script.
• examples/rag-example/: End-to-end RAG pipeline using the Qdrant vector database and Ollama embeddings.

Every pull request runs an end-to-end test on GitHub Actions (amd64 and arm64): init, up, pull a very small model (smollm:135m, about 90 MB), list it via /v1/models, a non-streaming /v1/chat/completions request, rate limiting (401 and 429), and a check that Open WebUI's start page responds. Not covered yet: streaming responses, embeddings, GPU acceleration, and macOS/Windows hosts.

Also tested by hand on an Apple M1 Mac with Docker Desktop and Ollama from Homebrew (0.5.2): init with the minimal preset and native Ollama, up, signing in to Open WebUI, chatting with gemma3:1b, and opening pondok-ai share from a phone on the same Wi-Fi.

• Not tested on Windows hosts yet. macOS is tested by hand on Apple Silicon (see above), but not in CI.
• On Apple Silicon, Docker containers cannot use the GPU. Run pondok-ai init --native-ollama to use a natively installed Ollama with Metal instead; see the GPU guide.
• GPU acceleration, streaming responses, and embeddings are not covered by CI yet.
• The images are large (several GB). Plan for substantial free disk space; a 32 GB cloud workspace with 19 GB free ran out of space while extracting the Ollama image.

• Architecture Overview
• API Gateway Reference
• Configuration Guide
• GPU Acceleration Setup (Metal / CUDA)
• Troubleshooting & Diagnostics
• Recording the README demo

Contributions are welcome! Check out CONTRIBUTING.md to get started.

MIT © 2026 pondok-ai contributors