diff --git a/graphile/graphile-llm/README.md b/graphile/graphile-llm/README.md index 364076bc7..03816aac6 100644 --- a/graphile/graphile-llm/README.md +++ b/graphile/graphile-llm/README.md @@ -1,14 +1,34 @@ # graphile-llm -LLM integration plugin for PostGraphile v5. Provides server-side text-to-vector embedding for pgvector columns. +

+ +

-## Features +

+ + + + + +

+ +LLM integration plugin for PostGraphile v5 — server-side text-to-vector embedding, resolve-time vector injection, and RAG (Retrieval-Augmented Generation) for pgvector columns using `@agentic-kit/ollama`. + +## Table of contents + +- [Installation](#installation) +- [Usage](#usage) +- [Features](#features) +- [Plugins](#plugins) +- [Configuration](#configuration) +- [RAG queries](#rag-queries) +- [License](#license) -- **Text-based vector search**: Adds `text: String` field to `VectorNearbyInput` — clients pass natural language instead of raw float vectors -- **Text mutation fields**: Adds `{column}Text: String` companion fields on create/update inputs for vector columns -- **Pluggable embedders**: Provider-based architecture (Ollama via `@agentic-kit/ollama`, with room for OpenAI, etc.) -- **Per-database configuration**: Reads `llm_module` from `services_public.api_modules` for per-API embedder config -- **Plugin-conditional**: Fields only appear in the schema when the plugin is loaded +## Installation + +```bash +npm install graphile-llm +``` ## Usage @@ -27,3 +47,89 @@ const preset = { ], }; ``` + +The preset bundles all plugins listed below. You can also import each plugin individually (`createLlmModulePlugin`, `createLlmTextSearchPlugin`, `createLlmTextMutationPlugin`, `createLlmRagPlugin`) if you prefer a-la-carte. + +## Features + +- **Text-based vector search** — adds `text: String` field to `VectorNearbyInput`; clients pass natural language instead of raw float vectors +- **Text mutation fields** — adds `{column}Text: String` companion fields on create/update inputs for vector columns +- **RAG queries** — adds `ragQuery` and `embedText` root query fields; detects `@hasChunks` smart tags for chunk-aware retrieval +- **Pluggable providers** — provider-based architecture for both embedding and chat completion (Ollama via `@agentic-kit/ollama`, extensible to OpenAI, etc.) +- **Per-database configuration** — reads `llm_module` from `services_public.api_modules` for per-API provider config +- **Toggleable** — each capability (`enableTextSearch`, `enableTextMutations`, `enableRag`) can be independently enabled or disabled +- **Plugin-conditional** — fields only appear in the schema when the plugin is loaded + +## Plugins + +| Plugin | Description | Toggle | +|--------|-------------|--------| +| `LlmModulePlugin` | Resolves embedder and chat completer from config; stores on build context | Always included | +| `LlmTextSearchPlugin` | Adds `text: String` to `VectorNearbyInput` with resolve-time embedding | `enableTextSearch` (default: `true`) | +| `LlmTextMutationPlugin` | Adds `{column}Text` companion fields on mutation inputs | `enableTextMutations` (default: `true`) | +| `LlmRagPlugin` | Adds `ragQuery` and `embedText` root query fields | `enableRag` (default: `false`) | + +## Configuration + +```typescript +GraphileLlmPreset({ + // Embedding provider (required for text fields and RAG) + defaultEmbedder: { + provider: 'ollama', + model: 'nomic-embed-text', + baseUrl: 'http://localhost:11434', + }, + + // Chat completion provider (required for RAG) + defaultChatCompleter: { + provider: 'ollama', + model: 'llama3', + baseUrl: 'http://localhost:11434', + }, + + // Toggle individual capabilities + enableTextSearch: true, // text field on VectorNearbyInput + enableTextMutations: true, // *Text companion fields on mutations + enableRag: false, // ragQuery + embedText root fields + + // RAG defaults (overridable per-query) + ragDefaults: { + contextLimit: 10, + maxTokens: 4000, + minSimilarity: 0.3, + }, +}) +``` + +Providers can also be configured via environment variables (`EMBEDDER_PROVIDER`, `EMBEDDER_MODEL`, `EMBEDDER_BASE_URL`, `CHAT_PROVIDER`, `CHAT_MODEL`, `CHAT_BASE_URL`). + +## RAG queries + +When `enableRag: true` and tables have `@hasChunks` smart tags, the plugin adds: + +```graphql +# Full RAG: embed prompt, search chunks, assemble context, call chat LLM +query { + ragQuery( + prompt: "What is machine learning?" + contextLimit: 5 + minSimilarity: 0.3 + ) { + answer + sources { content similarity tableName parentId } + tokensUsed + } +} + +# Standalone embedding +query { + embedText(text: "machine learning concepts") { + vector + dimensions + } +} +``` + +## License + +MIT