core(db) : Migration from mongodb to postgresql

[ijusttookadnatest]

Database Migration: MongoDB to PostgreSQL

Summary

This PR implements a comprehensive migration from MongoDB Atlas to PostgreSQL with pgvector extension for the Cairo Coder project. This change enhances our vector search capabilities while providing a more robust and standardized database solution.

Key Changes

• Database Engine: Replaced MongoDB Atlas with PostgreSQL (pgvector extension) for vector embeddings storage
• Configuration System: Updated configuration files to support PostgreSQL connection parameters
• Vector Store Implementation: Rewrote the VectorStore class to use PostgreSQL instead of MongoDB
• Docker Setup: Added PostgreSQL container to docker-compose.yml with proper networking
• Documentation: Updated README and project instructions with PostgreSQL configuration details

Technical Implementation

• Added two-tiered configuration approach:
  • .env in root directory (for PostgreSQL container initialization)
  • config.toml in packages/agents (for application database connection)
• Implemented singleton pattern for database connection management
• Added proper error handling for PostgreSQL-specific errors
• Optimized vector search performance with ivfflat index

Configuration Example

# .env file for container initialization
POSTGRES_USER="user"
POSTGRES_PASSWORD="secure_password"
POSTGRES_ROOT_DB="postgres"
POSTGRES_HOST="localhost"
POSTGRES_PORT="5432"

# config.toml for application connection
[VECTOR_DB]
POSTGRES_USER="user"
POSTGRES_PASSWORD="secure_password" 
POSTGRES_ROOT_DB="postgres"
POSTGRES_HOST="postgres"
POSTGRES_PORT="5432"

Testing

All RAG pipeline tests have been updated to work with the new PostgreSQL implementation. Vector similarity search functionality has been verified with multiple test cases.

Migration Path

For existing deployments, a migration script is provided to transfer data from MongoDB to PostgreSQL. New deployments can start directly with the PostgreSQL implementation.