feat(ai): Implement complete openAiCompatible provider architecture (#9766)

tobiu · tobiu · commit 30fb596043a9 · 2026-04-07T22:25:38.000+02:00
- Refactored TextEmbeddingService.mjs to support native /v1/embeddings targeting MLX endpoints using ES6 optional chaining
- Documented openAiCompatible architecture universally inside JSDocs and configurations
- Safely added apiKey endpoint mapping via standard dot-env security practices
- Finalized MLX/OpenAI compatibility inside AI quick start documentation
diff --git a/.github/AI_QUICK_START.md b/.github/AI_QUICK_START.md
@@ -83,7 +83,11 @@ To ensure your environment is accurately inherited across all local MCP sub-serv
     ```bash
     export GEMINI_API_KEY="YOUR_API_KEY_HERE"
     export GH_TOKEN="YOUR_GITHUB_TOKEN_HERE"
-    export NEO_EMBEDDING_PROVIDER="gemini" # or "ollama" for 100% offline
+    # Core LLM Engine provider (Supported: 'gemini', 'ollama', 'openAiCompatible')
+    # export MODEL_PROVIDER="openAiCompatible"
+    
+    # Vector Embedding provider (Supported: 'gemini', 'ollama', 'openAiCompatible')
+    export NEO_EMBEDDING_PROVIDER="gemini"
     ```
 3.  **Apply changes**: `source ~/.zshrc`
 
@@ -101,8 +105,8 @@ The free tier of the Gemini API has a strict limit of **1,000 requests per day**
 *   **Do NOT** run `npm run ai:sync-kb` (full rebuild) unless absolutely necessary. A full rebuild requires ~153 requests and takes ~25 minutes due to rate-limiting delays.
 *   The pre-built artifact saves you from this cost and delay.
 
-**Local Architecture (Ollama Integration):**
-To completely bypass Gemini API limits and operate a 100% offline knowledge base, you can utilize the local inference loop. Export `NEO_EMBEDDING_PROVIDER="ollama"` in your environment variables. Ensure the corresponding Ollama embedding models are running locally.
+**Local Architecture (Ollama & MLX / OpenAI-Compatible):**
+To completely bypass Gemini API limits and operate a 100% offline knowledge base, you can utilize the local inference loops. Export `NEO_EMBEDDING_PROVIDER="ollama"` or `NEO_EMBEDDING_PROVIDER="openAiCompatible"` in your environment variables. Ensure the corresponding embedding models and servers (e.g. `mlx_lm.server`) are running locally.
 
 ### Step 3.5: Advanced Configuration (Optional)
 
diff --git a/ai/mcp/server/memory-core/config.template.mjs b/ai/mcp/server/memory-core/config.template.mjs
@@ -63,17 +63,20 @@ const defaultConfig = {
     },
     /**
      * Explicit override provider for the core LLM Engine (e.g. summarization).
-     * @type {string}
+     * Supported values: 'gemini', 'ollama', 'openAiCompatible'
+     * @type {String}
      */
     modelProvider: process.env.MODEL_PROVIDER || 'gemini',
     /**
      * Explicit override provider for the SQLite Native Database Engine.
-     * @type {string}
+     * Supported values: 'gemini', 'ollama', 'openAiCompatible'
+     * @type {String}
      */
     neoEmbeddingProvider: process.env.NEO_EMBEDDING_PROVIDER || 'gemini',
     /**
      * Explicit override provider for the ChromaDB Engine.
-     * @type {string}
+     * Supported values: 'gemini', 'ollama', 'openAiCompatible'
+     * @type {String}
      */
     chromaEmbeddingProvider: process.env.CHROMA_EMBEDDING_PROVIDER || 'gemini',
     /**
@@ -86,11 +89,13 @@ const defaultConfig = {
     },
     /**
      * Settings for the OpenAI-Compatible API integration (e.g., mlx-lm or mlx-openai-server)
+     * WARNING: Never hardcode API keys here. Always export them via .env or globally.
      */
     openAiCompatible: {
         host          : process.env.OPENAI_COMPATIBLE_HOST || 'http://127.0.0.1:8000',
         model         : process.env.OPENAI_COMPATIBLE_MODEL || 'gemma4:31b',
-        embeddingModel: process.env.OPENAI_COMPATIBLE_EMBEDDING_MODEL || 'qwen3-embedding'
+        embeddingModel: process.env.OPENAI_COMPATIBLE_EMBEDDING_MODEL || 'qwen3-embedding',
+        apiKey        : process.env.OPENAI_COMPATIBLE_API_KEY || ''
     },
     /**
      * The name of the Google Generative AI model for content generation.
diff --git a/ai/mcp/server/memory-core/services/TextEmbeddingService.mjs b/ai/mcp/server/memory-core/services/TextEmbeddingService.mjs
@@ -108,6 +108,59 @@ class TextEmbeddingService extends Base {
                 logger.error(`[TextEmbeddingService] Failed to generate embedding from Ollama:`, err.message);
                 throw err;
             }
+        } else if (explicitProvider === 'openAiCompatible') {
+            const { host, embeddingModel, apiKey } = aiConfig.openAiCompatible;
+            try {
+                const parsedUrl = new URL(`${host}/v1/embeddings`);
+                const httpModule = parsedUrl.protocol === 'https:' ? await import('https') : await import('http');
+
+                let resolveFunc, rejectFunc;
+                const responsePromise = new Promise((res, rej) => {
+                    resolveFunc = res;
+                    rejectFunc = rej;
+                });
+
+                const reqHeaders = { 'Content-Type': 'application/json' };
+                if (apiKey) {
+                    reqHeaders.Authorization = `Bearer ${apiKey}`;
+                }
+
+                const req = httpModule.request(parsedUrl, {
+                    method : 'POST',
+                    headers: reqHeaders,
+                    timeout: 60 * 60 * 1000 // 1 hour timeout natively
+                }, (res) => {
+                    let body = '';
+                    res.on('data', chunk => body += chunk);
+                    res.on('end', () => {
+                        if (res.statusCode < 200 || res.statusCode >= 300) {
+                            rejectFunc(new Error(`openAiCompatible embedding error HTTP ${res.statusCode}: ${body}`));
+                        } else {
+                            try {
+                                const result = JSON.parse(body);
+                                resolveFunc(result);
+                            } catch (e) {
+                                rejectFunc(new Error(`Failed to parse openAiCompatible response: ${e.message}`));
+                            }
+                        }
+                    });
+                });
+
+                req.on('error', (err) => rejectFunc(err));
+                req.on('timeout', () => {
+                    req.destroy();
+                    rejectFunc(new Error('openAiCompatible request timed out after 1 hour'));
+                });
+
+                req.write(JSON.stringify({ model: embeddingModel, input: text }));
+                req.end();
+
+                const result = await responsePromise;
+                return result.data?.[0]?.embedding;
+            } catch (err) {
+                logger.error(`[TextEmbeddingService] Failed to generate embedding from openAiCompatible:`, err.message);
+                throw err;
+            }
         } else {
             if (!process.env.GEMINI_API_KEY) {
                  throw new Error('Semantic search unavailable: GEMINI_API_KEY is missing.');
diff --git a/resources/content/sandman_handoff.md b/resources/content/sandman_handoff.md
@@ -34,3 +34,6 @@ Based on the latest Tri-Vector Synthesis and Topological Priorities, the followi
 - **[Codebase Gap]** Node `Ticket #8462`: [DOC_GAP] No files related to 'SectionsList' were found in the directory tree. There is a complete lack of evidence for existing documentation or test coverage regarding the styling and implementation of the SectionsList component. (Source Session: fd1a1232-8923-469d-990b-73b00efd1ef1)
 - **[Codebase Gap]** Node `.neo-list-item`: [DOC_GAP] The CSS class `.neo-list-item` intended for `SectionsList` is not documented in the `docs/` or `learn/` directories, and there are no corresponding tests in `test/playwright/` to verify the rendering or styling of these list items. (Source Session: fd1a1232-8923-469d-990b-73b00efd1ef1)
 - **[Codebase Gap]** Node `Issue #8620`: [DOC_GAP] No functional tests or documentation found for Element.moveBefore support. The provided directory tree contains only a workflow test file (OpenapiIssues.spec.mjs), which does not cover the implementation or usage of the Element.moveBefore method. (Source Session: fd130b21-2cfe-4230-b3ba-4d2cd527d2d2)
+- **[Codebase Gap]** Node `src/state/Provider.mjs`: [DOC_GAP] While comprehensive guides exist for reactivity and usage, there is no documentation regarding 'store serialization' specifically mentioned in the node description. Additionally, no corresponding test files were found in the provided directory tree for src/state/Provider.mjs. (Source Session: fd099a94-e4dc-4780-ab47-f2f91685aaf2)
+- **[Codebase Gap]** Node `src/layout/Base.mjs`: [DOC_GAP] src/layout/Base.mjs (Base layout class) lacks native test coverage and developer documentation. While 'learn/guides/uibuildingblocks/Layouts.md' exists, it is a user guide for applying layouts via configuration and does not document the internal serialization logic or the Base class API for developers. (Source Session: fd099a94-e4dc-4780-ab47-f2f91685aaf2)
+- **[Codebase Gap]** Node `src/layout/Flexbox.mjs`: [DOC_GAP] The 'gap' property mentioned in the Flexbox.mjs description (prefix and gap serialization) is missing from the documentation in 'learn/guides/uibuildingblocks/Layouts.md', which only covers basic align and pack properties for vbox/hbox. Additionally, there are no corresponding test files for the layout implementation in the provided directory tree. (Source Session: fd099a94-e4dc-4780-ab47-f2f91685aaf2)
