Feature request: Flag to enable RAG

[leseb]

By cooktheryan in opendatahub-io#18

The operator should have the ability to optionally deploy Milvus-lite as the vectorDB. This can be accomplished by adding the following to the llama stack run config

  vector_io:
  - provider_id: milvus
    provider_type: inline::milvus
    config:
     db_path: ${env.MILVUS_DB_PATH}

The DB path we could either make an optional path or hardcode something that we can attach a pvc to

[liangwen12year]

User Story: RAG (Vector Database) Support via Milvus
As a platform engineer deploying the Llama Stack K8s Operator,
I want the ability to enable RAG by specifying a vector database (such as Milvus) in the LlamaStackDistribution CR,
so that I can easily deploy Llama Stack with integrated vector search capabilities, and optionally configure persistent storage to retain vector data across pod restarts.

Acceptance Criteria:

1. ✅CRD Extension:
   The LlamaStackDistribution CR supports a new rag section with the following fields:

• enabled (boolean, required): enables or disables RAG.
• provider (string, optional, default: milvus): specifies the vector DB provider.
• pvcName (string, optional): names the PersistentVolumeClaim to attach for storage.
• dbPath (string, optional, default: $HOME/.llama/milvus.db): specifies the vector DB file path.

2. ✅Config Injection:
   When rag.enabled is set to true, the Operator injects the appropriate vector_io configuration (e.g., Milvus) into the Llama Stack runtime config.

3. ✅Environment Variables:
   The Operator sets the required environment variable(s) for the vector DB, such as MILVUS_DB_PATH, matching the configured or default path.

4. ✅Storage Management:

• If pvcName is provided, the Operator mounts the specified PVC at the directory containing dbPath.

• If pvcName is not provided, the Operator mounts an ephemeral emptyDir volume at the parent directory of dbPath to support RAG functionality without persistence.

5. ✅Deployment Health:
   The Operator ensures that the Llama Stack deployment starts successfully and that the vector DB is accessible when RAG is enabled.

6. ✅Status Reporting:
   The Operator updates the LlamaStackDistribution CR status to include:

• Whether RAG is enabled.

• The current provider (e.g., milvus).

• The mounted storage info (PVC name and mount path).

• The health status of the vector DB (e.g., Ready, Error).

7. ✅Documentation:

• CR examples are provided showing how to enable RAG with and without a custom PVC.

• Clear documentation explains default behaviors (e.g., provider defaults to Milvus, emptyDir fallback).

[VaishnaviHire]

@leseb Is this issue still needed?

[leseb]

@leseb Is this issue still needed?

No when the CM stuff gets in.