VectorSearch RAG Service

This is a production-ready Retrieval-Augmented Generation (RAG) service. It uses a Go API for querying and a Python worker for creating embeddings, with PostgreSQL (using pgvector) for storage and Redis for caching and queuing.

Motivation

Imagine a new engineer joining your team, faced with a mountain of onboarding documents, wikis, and design specs. Finding answers to specific questions becomes a frustrating treasure hunt, as keyword search often misses the context behind a query.

This project was built to solve that problem. vectorSearch turns chaotic internal documentation into a smart, searchable knowledge base. By indexing content and using vector search, it allows developers to ask natural language questions and get precise, context-aware answers, dramatically speeding up onboarding and knowledge discovery.

Quick Start

Prerequisites: Docker & Docker Compose

1. Clone and Setup:

   git clone <repo-url>
   cd vectorSearch
   cp .env.example .env 
   # Add your OpenAI API key to .env if you have one

2. Start Services:

   make up
   # Or: cd deploy && docker-compose up -d --build

3. Seed Data & Query:

   make seed
   ./scripts/curl_examples.sh

Key Components

• Go Query API (/cmd/api): Handles search queries.
• Python Embeddings Worker (/python_worker): Generates and indexes document embeddings.
• PostgreSQL + pgvector: Stores vector embeddings.
• Redis: Used for caching and as a message queue.
• Docker Compose (/deploy): For local development environment.

API Endpoints

• POST /index: Enqueue a document for embedding.
• POST /query: Search for similar documents.
• GET /healthz: Health check.
• GET /metrics: Prometheus metrics.

Local Development

To run services without Docker:

1. Install dependencies: go mod download and pip install -r python_worker/requirements.txt.
2. Ensure PostgreSQL and Redis are running.
3. Run the services:
   • make dev-api
   • make dev-fastapi
   • make dev-worker