Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support new backend implementation for rag-ai plugin #1382

Open
tranngoclam opened this issue May 8, 2024 · 1 comment
Open

Support new backend implementation for rag-ai plugin #1382

tranngoclam opened this issue May 8, 2024 · 1 comment
Labels
kind/enhancement New feature or request

Comments

@tranngoclam
Copy link

Feature Suggestion

  • Support new backend of backstage for rag-ai plugin

Possible Implementation

N/A

Context

I have added rag-ai plugin and used with new backend of backstage, but got an error during startup

2024-05-08T04:17:34.370765253Z /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1806
2024-05-08T04:17:34.370773628Z           throw new errors.ForwardedError(
2024-05-08T04:17:34.370776545Z                 ^
2024-05-08T04:17:34.370789462Z
2024-05-08T04:17:34.370791337Z ForwardedError: Plugin 'rag-ai' startup failed; caused by Error: augmentationIndexer must be registered
2024-05-08T04:17:34.370792753Z     at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1806:17
2024-05-08T04:17:34.370794170Z     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-05-08T04:17:34.370795545Z     at async /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1805:9
2024-05-08T04:17:34.370797045Z     at async Promise.all (index 9)
2024-05-08T04:17:34.370798378Z     ... 2 lines matching cause stack trace ...
2024-05-08T04:17:34.370799712Z     at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1894:5) {
2024-05-08T04:17:34.370809295Z   cause: Error: augmentationIndexer must be registered
2024-05-08T04:17:34.370811170Z       at Object.init [as func] (/app/node_modules/@roadiehq/rag-ai-backend/dist/index.cjs.js:255:17)
2024-05-08T04:17:34.370813086Z       at /app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1805:31
2024-05-08T04:17:34.370814795Z       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2024-05-08T04:17:34.370816420Z       at async Promise.all (index 9)
2024-05-08T04:17:34.370818170Z       at async BackendInitializer.doStart_fn (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1766:3)
2024-05-08T04:17:34.370819920Z       at async BackendInitializer.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1626:5)
2024-05-08T04:17:34.370821378Z       at async BackstageBackend.start (/app/node_modules/@backstage/backend-app-api/dist/index.cjs.js:1894:5)
2024-05-08T04:17:34.370822836Z }
2024-05-08T04:17:34.370824086Z

My implementation

packages/backend/src/index.ts

import {legacyPlugin} from '@backstage/backend-common';
import {createBackend} from '@backstage/backend-defaults';

const backend = createBackend();

...
backend.add(legacyPlugin('ai', import('./plugins/ai')));
backend.add(import('@roadiehq/rag-ai-backend'));

packages/backend/src/plugins/ai.ts

import { createApiRoutes as initializeRagAiBackend } from '@roadiehq/rag-ai-backend';
import { initializeOpenAiEmbeddings } from '@roadiehq/rag-ai-backend-embeddings-openai';
import { createRoadiePgVectorStore } from '@roadiehq/rag-ai-storage-pgvector';
import { createDefaultRetrievalPipeline } from '@roadiehq/rag-ai-backend-retrieval-augmenter';
import { OpenAI } from '@langchain/openai';
import { CatalogClient } from '@backstage/catalog-client';
import { PluginEnvironment } from '../types';

export default async function createPlugin({
    logger,
    tokenManager,
    database,
    discovery,
    config,
}: PluginEnvironment) {
    const catalogApi = new CatalogClient({
        discoveryApi: discovery,
    });

    const vectorStore = await createRoadiePgVectorStore({ logger, database, config });

    const augmentationIndexer = await initializeOpenAiEmbeddings({
        logger,
        tokenManager,
        vectorStore,
        catalogApi,
        discovery,
        config,
    });

    const model = new OpenAI();

    const ragAi = await initializeRagAiBackend({
        logger,
        tokenManager,
        augmentationIndexer,
        retrievalPipeline: createDefaultRetrievalPipeline({
            discovery,
            logger,
            vectorStore: augmentationIndexer.vectorStore,
            tokenManager,
        }),
        model,
        config,
    });

    return ragAi.router;
}

package.json

  "dependencies": {
    "@backstage/backend-common": "^0.21.7",
    "@backstage/backend-defaults": "^0.2.17",
    "@backstage/backend-plugin-api": "^0.6.17",
    "@backstage/backend-tasks": "^0.5.22",
    "@backstage/catalog-client": "^1.6.4",
    "@backstage/catalog-model": "^1.4.5",
    "@backstage/config": "^1.2.0",
    "@backstage/plugin-app-backend": "^0.3.65",
    "@backstage/plugin-auth-backend": "^0.22.4",
    "@backstage/plugin-auth-backend-module-guest-provider": "^0.1.3",
    "@backstage/plugin-auth-node": "^0.4.12",
    "@backstage/plugin-catalog-backend": "^1.21.1",
    "@backstage/plugin-catalog-backend-module-gitlab": "^0.3.15",
    "@backstage/plugin-catalog-backend-module-gitlab-org": "^0.0.0-nightly-20240423021331",
    "@backstage/plugin-catalog-backend-module-scaffolder-entity-model": "^0.1.15",
    "@backstage/plugin-kubernetes-backend": "^0.17.0",
    "@backstage/plugin-permission-backend": "^0.5.41",
    "@backstage/plugin-permission-backend-module-allow-all-policy": "^0.1.14",
    "@backstage/plugin-permission-common": "^0.7.13",
    "@backstage/plugin-permission-node": "^0.7.28",
    "@backstage/plugin-proxy-backend": "^0.4.15",
    "@backstage/plugin-scaffolder-backend": "^1.22.5",
    "@backstage/plugin-search-backend": "^1.5.7",
    "@backstage/plugin-search-backend-module-catalog": "^0.1.23",
    "@backstage/plugin-search-backend-module-pg": "^0.5.26",
    "@backstage/plugin-search-backend-module-techdocs": "^0.1.22",
    "@backstage/plugin-search-backend-node": "^1.2.21",
    "@backstage/plugin-techdocs-backend": "^1.10.4",
    "@roadiehq/rag-ai-backend": "^0.3.0",
    "@roadiehq/rag-ai-backend-embeddings-openai": "^0.2.2",
    "@roadiehq/rag-ai-backend-retrieval-augmenter": "^0.3.0",
    "@roadiehq/rag-ai-storage-pgvector": "^0.1.1",
    "app": "link:../app",
    "better-sqlite3": "^8.0.0",
    "dockerode": "^3.3.1",
    "express": "^4.17.1",
    "express-promise-router": "^4.1.0",
    "pg": "^8.3.0",
    "winston": "^3.2.1"
  },
@tranngoclam tranngoclam added the kind/enhancement New feature or request label May 8, 2024
@kevinmartin
Copy link

This configuration was able to get me to startup, and queries OpenAI, but there is an auth error when querying the search endpoint for RAG entries.

// rag ai plugin
backend.add(import('@roadiehq/rag-ai-backend'));
backend.add(
  createBackendModule({
    pluginId: 'rag-ai',
    moduleId: 'config',
    register(env) {
      env.registerInit({
        deps: {
          tokenManager: coreServices.tokenManager,
          logger: coreServices.logger,
          database: coreServices.database,
          discovery: coreServices.discovery,
          config: coreServices.rootConfig,
          indexer: augmentationIndexerExtensionPoint,
          pipeline: retrievalPipelineExtensionPoint,
          model: modelExtensionPoint,
        },
        async init({ tokenManager, logger, database, discovery, config, indexer, pipeline, model }) {
          const catalogApi = new CatalogClient({ discoveryApi: discovery });
          const vectorStore = await createRoadiePgVectorStore({
            logger: loggerToWinstonLogger(logger),
            database,
            config,
          });

          indexer.setAugmentationIndexer(await initializeOpenAiEmbeddings({
            tokenManager,
            logger: loggerToWinstonLogger(logger),
            discovery,
            catalogApi,
            vectorStore,
            config,
          }));

          pipeline.setRetrievalPipeline(createDefaultRetrievalPipeline({
            tokenManager,
            logger: loggerToWinstonLogger(logger),
            discovery,
            vectorStore,
          }));

          model.setBaseLLM(new OpenAI({
            model: 'gpt-4o',
            temperature: 0.9,
            apiKey: config.getString('ai.embeddings.openai.openAIApiKey'),
          }));
        },
      });
    },
  })
);

Logged request error:

2024-06-06T12:01:00.847Z rag-ai info Received 0 embeddings from Vector store 
2024-06-06T12:01:00.851Z rootHttpRouter info ::ffff:127.0.0.1 - - [06/Jun/2024:12:01:00 +0000] "GET /api/search/query?term=Hello%20world&types[0]=software-catalog HTTP/1.1" 401 812 "-" "node" type=incomingRequest
2024-06-06T12:01:00.851Z rag-ai warn Unable to query Backstage search API for embeddable results. label=rag-ai-searchclient
2024-06-06T12:01:00.852Z rag-ai info Received 0 results when querying augmentations from search. 
2024-06-06T12:01:00.852Z rag-ai info Starting to prompt LLM. 
2024-06-06T12:01:01.667Z rootHttpRouter info ::ffff:127.0.0.1 - - [06/Jun/2024:12:01:01 +0000] "POST /backstage/api/rag-ai/query/catalog HTTP/1.1" 200 89 "[http://localhost:3000/"](http://localhost:3000/%22) "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36" type=incomingRequest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants