llama stack vector_store operations in lightspeed. Enhancing URL fetc… #2742

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

JslYoon wants to merge 6 commits into redhat-developer:main from JslYoon:JslYoon-notebooksPermissions

+1,938 −1,146

workspaces/lightspeed/app-config.yaml

-Original file line number
+Diff line change
@@ Expand Up / @@ -18,8 +18,15 @@ organization: @@
     # Disable AI Notebooks feature by default
     lightspeed:
-      aiNotebooks:
+      Notebooks:
         enabled: false
+        queryDefaults:
+          model: redhataillama-31-8b-instruct
+          provider_id: vllm
+        sessionDefaults:
+          provider_id: notebooks
+          embedding_model: ${LLAMA_STACK_EMBEDDING_MODEL}
+          embedding_dimension: 768
     backend:
       # Used for enabling authentication, secret is shared by all backend plugins
@@ Expand Down @@

workspaces/lightspeed/plugins/lightspeed-backend/README.md

            
                      Original file line number
                      Diff line number
                      Diff line change
                  
    @@ -89,7 +89,10 @@ For user-facing feature documentation, see the [Lightspeed Frontend README](../l
  
    #### Prerequisites

    AI Notebooks requires a **Llama Stack service** to be running. Llama Stack provides the vector database, embeddings, and RAG capabilities.

    AI Notebooks requires:

    - **Lightspeed Core service** to be running (provides the backend API proxy)

    - **Llama Stack service** to be accessible from Lightspeed Core (provides vector database, embeddings, and RAG capabilities)

    For Llama Stack setup and configuration, refer to the [Llama Stack documentation](https://github.com/llamastack/llama-stack).

    @@ -99,21 +102,23 @@ To enable AI Notebooks, add the following configuration to your `app-config.yaml
  
    ```yaml

    lightspeed:

      aiNotebooks:

        enabled: true # Enable AI Notebooks feature (default: false)

      servicePort: 8080 # Optional: Lightspeed Core service port (default: 8080)

        # Required when enabled: Llama Stack service configuration

        llamaStack:

          port: 8321 # Llama Stack API endpoint (required, commonly 8321)

      Notebooks:

        enabled: false # Enable AI Notebooks feature (default: false)

          # Optional embedding configuration

          embeddingModel: sentence-transformers/nomic-ai/nomic-embed-text-v1.5 # (default shown)

          embeddingDimension: 768 # Embedding vector dimension (default: 768)

          vectorIo:

            providerId: rhdh-docs # Vector store provider ID (default: rhdh-docs)

        # Required: Query defaults for RAG queries

        # Both model and provider_id must be configured together

        queryDefaults:

          model: llama3.1-8b-instruct # Model to use for answering queries

          provider_id: ollama # AI provider for the query model

        # Optional: File processing timeout (default: 30000ms = 30 seconds)

        fileProcessingTimeoutMs: 30000

        # Required: Session defaults for creating vector stores

        # All three fields are required when Notebooks is enabled

        sessionDefaults:

          provider_id: notebooks # Vector store provider ID (must match Llama Stack config)

          embedding_model: sentence-transformers/all-mpnet-base-v2 # Model for generating embeddings

          embedding_dimension: 768 # Embedding vector dimension (must match model output)

        # Optional: Chunking strategy for document processing

        chunkingStrategy:

    @@ -125,15 +130,37 @@ lightspeed:
  
    **Configuration Options**:

    - **`enabled`**: Enable or disable the AI Notebooks feature (default: `false`)

    - **`llamaStack.port`**: Port of the Llama Stack service (default: `8321`)

    - **`llamaStack.embeddingModel`**: Model used for generating embeddings (default: `sentence-transformers/nomic-ai/nomic-embed-text-v1.5`)

    - **`llamaStack.embeddingDimension`**: Dimension of embedding vectors (default: `768`)

    - **`llamaStack.vectorIo.providerId`**: Vector store provider in Llama Stack config (default: `rhdh-docs`)

    - **`fileProcessingTimeoutMs`**: Timeout for file processing in milliseconds (default: `30000`)

    - **`chunkingStrategy.type`**: Document chunking strategy - `auto` (automatic) or `static` (fixed size) (default: `auto`)

    - **`chunkingStrategy.maxChunkSizeTokens`**: Maximum chunk size in tokens for static chunking (default: `512`)

    - **`chunkingStrategy.chunkOverlapTokens`**: Token overlap between chunks for static chunking (default: `50`)

    **Core Settings**:

    - **`lightspeed.servicePort`** _(optional)_: Port where Lightspeed Core service is running (default: `8080`). The backend connects to Lightspeed Core at `http://0.0.0.0:{servicePort}` to proxy vector store operations.

    **Notebooks Settings**:

    - **`Notebooks.enabled`** _(optional)_: Enable or disable the AI Notebooks feature (default: `false`)

    **Query Defaults** _(required when enabled)_:

    - **`queryDefaults.model`** _(required)_: The LLM model to use for answering RAG queries. Must be available in the configured provider.

    - **`queryDefaults.provider_id`** _(required)_: The AI provider identifier for the query model (e.g., `ollama`, `vllm`). Both `model` and `provider_id` must be configured together.

    **Session Defaults** _(required when enabled)_:

    - **`sessionDefaults.provider_id`** _(required)_: Vector store provider identifier. Must match a provider configured in your Llama Stack instance (e.g., `notebooks`, `chromadb`). This determines where document embeddings are stored.

    - **`sessionDefaults.embedding_model`** _(required)_: The embedding model to use for converting documents to vectors (e.g., `sentence-transformers/all-mpnet-base-v2`). Must be available in Llama Stack.

    - **`sessionDefaults.embedding_dimension`** _(required)_: Dimension of the embedding vectors produced by the embedding model. Must match the model's output dimension (commonly `768`, `384`, or `1536`).

    **Chunking Strategy** _(optional)_:

    - **`chunkingStrategy.type`** _(optional)_: Document chunking strategy - `auto` (automatic, default) or `static` (fixed size)

    - **`chunkingStrategy.maxChunkSizeTokens`** _(optional)_: Maximum chunk size in tokens for static chunking (default: `512`)

    - **`chunkingStrategy.chunkOverlapTokens`** _(optional)_: Token overlap between chunks for static chunking (default: `50`)

    **Where to Find These Values**:

    - **Provider IDs**: Check your Llama Stack configuration file for configured providers (both for models and vector stores)

    - **Model names**: Available models are listed in your Llama Stack provider configuration

    - **Embedding dimensions**: Refer to the embedding model's documentation (e.g., `all-mpnet-base-v2` outputs 768 dimensions)

    - **Lightspeed Core port**: Check your Lightspeed Core service deployment configuration

    #### API Endpoints

    @@ -144,20 +171,26 @@ When enabled, AI Notebooks exposes the following REST API endpoints:
  
    - **Sessions**:

      - `POST /lightspeed/ai-notebooks/v1/sessions` - Create a new session

      - `GET /lightspeed/ai-notebooks/v1/sessions` - List all sessions

      - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Get session details

      - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Update session

      - `GET /lightspeed/ai-notebooks/v1/sessions` - List all sessions for the current user

      - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Update session details

      - `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Delete session

    - **Documents**:

      - `POST /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/upload` - Upload document

      - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents` - List documents

      - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Update document

      - `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Delete document

      - `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents` - Upload or update a document (multipart/form-data)

      - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents` - List all documents in a session

      - `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId/status` - Get document processing status

      - `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Delete a document

    - **Queries**:

      - `POST /lightspeed/ai-notebooks/v1/sessions/:sessionId/query` - Query documents with RAG

    **Notes**:

    - All endpoints require authentication (user context is automatically provided by Backstage)

    - All `/v1/*` endpoints require the `lightspeed.notebooks.use` permission

    - Document endpoints verify session ownership before allowing operations

    - `documentId` in paths is the document title (URL-encoded for special characters)

    #### Permission Framework Support for AI Notebooks

    When RBAC is enabled, users need the following permission to use AI Notebooks:

workspaces/lightspeed/plugins/lightspeed-backend/__fixtures__/lightspeedCoreHandlers.ts

-Original file line number
+Diff line change
@@ -0,0 +1,237 @@
+    /*
+     * Copyright Red Hat, Inc.
+     *
+     * Licensed under the Apache License, Version 2.0 (the "License");
+     * you may not use this file except in compliance with the License.
+     * You may obtain a copy of the License at
+     *
+     *     http://www.apache.org/licenses/LICENSE-2.0
+     *
+     * Unless required by applicable law or agreed to in writing, software
+     * distributed under the License is distributed on an "AS IS" BASIS,
+     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+     * See the License for the specific language governing permissions and
+     * limitations under the License.
+     */
+    import { http, HttpResponse, type HttpHandler } from 'msw';
+    import { DEFAULT_LIGHTSPEED_SERVICE_PORT } from '../src/service/constant';
+    export const LIGHTSPEED_CORE_ADDR = `http://0.0.0.0:${DEFAULT_LIGHTSPEED_SERVICE_PORT}`;
+    // Mock session data
+    export const mockSession1 = {
+      session_id: 'session-1',
+      name: 'Test Session 1',
+      user_id: 'user:default/guest',
+      description: 'Test description',
+      created_at: '2024-01-01T00:00:00.000Z',
+      updated_at: '2024-01-01T00:00:00.000Z',
+      metadata: {
+        embedding_model: 'sentence-transformers/nomic-ai/nomic-embed-text-v1.5',
+        embedding_dimension: 768,
+        provider_id: 'notebooks',
+        conversation_id: null,
+      },
+    };
+    export const mockSession2 = {
+      session_id: 'session-2',
+      name: 'Test Session 2',
+      user_id: 'user:default/guest',
+      description: 'Another test',
+      created_at: '2024-01-02T00:00:00.000Z',
+      updated_at: '2024-01-02T00:00:00.000Z',
+      metadata: {
+        embedding_model: 'sentence-transformers/nomic-ai/nomic-embed-text-v1.5',
+        embedding_dimension: 768,
+        provider_id: 'notebooks',
+        conversation_id: 'conv-1',
+      },
+    };
+    export const mockFile1 = {
+      id: 'file-1',
+      created_at: 1704067200,
+      status: 'completed' as const,
+      attributes: {
+        document_id: 'test-document',
+        user_id: 'user:default/guest',
+        title: 'Test Document',
+        session_id: 'session-1',
+        source_type: 'text',
+        created_at: '2024-01-01T00:00:00.000Z',
+      },
+    };
+    // In-memory storage for tests
+    const vectorStores = new Map<string, any>();
+    const files = new Map<string, any>();
+    const vectorStoreFiles = new Map<string, any[]>();
+    export function resetMockStorage() {
+      vectorStores.clear();
+      files.clear();
+      vectorStoreFiles.clear();
+    }
+    /**
+     * MSW handlers for lightspeed-core vector store endpoints
+     * These mock the endpoints created in lightspeed-core that proxy to llama stack
+     */
+    export const lightspeedCoreHandlers: HttpHandler[] = [
+      // Create vector store
+      http.post(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores`, async ({ request }) => {
+        const body = (await request.json()) as any;
+        const id = `vs-${Date.now()}`;
+        const vectorStore = {
+          id,
+          name: body.name,
+          embedding_model: body.embedding_model,
+          embedding_dimension: body.embedding_dimension,
+          provider_id: body.provider_id,
+          metadata: body.metadata || {},
+        };
+        vectorStores.set(id, vectorStore);
+        vectorStoreFiles.set(id, []);
+        return HttpResponse.json(vectorStore);
+      }),
+      // Get vector store
+      http.get(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id`, ({ params }) => {
+        const { id } = params;
+        const vectorStore = vectorStores.get(id as string);
+        if (!vectorStore) {
+          return HttpResponse.json(
+            { detail: 'Vector store not found' },
+            { status: 404 },
+          );
+        }
+        return HttpResponse.json(vectorStore);
+      }),
+      // Update vector store
+      http.put(
+        `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id`,
+        async ({ params, request }) => {
+          const { id } = params;
+          const vectorStore = vectorStores.get(id as string);
+          if (!vectorStore) {
+            return HttpResponse.json(
+              { detail: 'Vector store not found' },
+              { status: 404 },
+            );
+          }
+          const body = (await request.json()) as any;
+          const updated = {
+            ...vectorStore,
+            metadata: body.metadata || vectorStore.metadata,
+          };
+          vectorStores.set(id as string, updated);
+          return HttpResponse.json(updated);
+        },
+      ),
+      // Delete vector store
+      http.delete(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id`, ({ params }) => {
+        const { id } = params;
+        if (!vectorStores.has(id as string)) {
+          return HttpResponse.json(
+            { detail: 'Vector store not found' },
+            { status: 404 },
+          );
+        }
+        vectorStores.delete(id as string);
+        vectorStoreFiles.delete(id as string);
+        return HttpResponse.json({ deleted: true });
+      }),
+      // List vector stores
+      http.get(`${LIGHTSPEED_CORE_ADDR}/v1/vector-stores`, () => {
+        const data = Array.from(vectorStores.values());
+        return HttpResponse.json({ data });
+      }),
+      // Upload file
+      http.post(`${LIGHTSPEED_CORE_ADDR}/v1/files`, async () => {
+        const fileId = `file-${Date.now()}`;
+        const file = {
+          id: fileId,
+          created_at: Date.now(),
+          purpose: 'assistants',
+        };
+        files.set(fileId, file);
+        return HttpResponse.json(file);
+      }),
+      // Add file to vector store
+      http.post(
+        `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files`,
+        async ({ params, request }) => {
+          const { id } = params;
+          const vectorStore = vectorStores.get(id as string);
+          if (!vectorStore) {
+            return HttpResponse.json(
+              { detail: 'Vector store not found' },
+              { status: 404 },
+            );
+          }
+          const body = (await request.json()) as any;
+          const vectorStoreFile = {
+            id: body.file_id,
+            status: 'completed' as const,
+            created_at: Date.now(),
+            chunks_count: 1,
+            attributes: body.attributes || {},
+          };
+          const storeFiles = vectorStoreFiles.get(id as string) || [];
+          storeFiles.push(vectorStoreFile);
+          vectorStoreFiles.set(id as string, storeFiles);
+          return HttpResponse.json(vectorStoreFile);
+        },
+      ),
+      // List files in vector store
+      http.get(
+        `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files`,
+        ({ params }) => {
+          const { id } = params;
+          const storeFiles = vectorStoreFiles.get(id as string) || [];
+          return HttpResponse.json({ data: storeFiles });
+        },
+      ),
+      // Get file from vector store
+      http.get(
+        `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files/:fileId`,
+        ({ params }) => {
+          const { id, fileId } = params;
+          const storeFiles = vectorStoreFiles.get(id as string) || [];
+          const file = storeFiles.find(f => f.id === fileId);
+          if (!file) {
+            return HttpResponse.json({ detail: 'File not found' }, { status: 404 });
+          }
+          return HttpResponse.json(file);
+        },
+      ),
+      // Delete file from vector store
+      http.delete(
+        `${LIGHTSPEED_CORE_ADDR}/v1/vector-stores/:id/files/:fileId`,
+        ({ params }) => {
+          const { id, fileId } = params;
+          const storeFiles = vectorStoreFiles.get(id as string) || [];
+          const fileIndex = storeFiles.findIndex(f => f.id === fileId);
+          if (fileIndex === -1) {
+            return HttpResponse.json({ detail: 'File not found' }, { status: 404 });
+          }
+          storeFiles.splice(fileIndex, 1);
+          vectorStoreFiles.set(id as string, storeFiles);
+          return HttpResponse.json({ deleted: true });
+        },
+      ),
+    ];

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

llama stack vector_store operations in lightspeed. Enhancing URL fetc… #2742

Diff view

Diff view

There are no files selected for viewing

SonarQubeCloud / SonarCloud Code Analysis

SonarQubeCloud / SonarCloud Code Analysis

SonarQubeCloud / SonarCloud Code Analysis

SonarQubeCloud / SonarCloud Code Analysis

Uh oh!

Uh oh!

llama stack vector_store operations in lightspeed. Enhancing URL fetc… #2742

Are you sure you want to change the base?

llama stack vector_store operations in lightspeed. Enhancing URL fetc… #2742

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

SonarQubeCloud / SonarCloud Code Analysis

SonarQubeCloud / SonarCloud Code Analysis

SonarQubeCloud / SonarCloud Code Analysis

SonarQubeCloud / SonarCloud Code Analysis

Uh oh!

Uh oh!