Semantic Search RAG API

Production-ready FastAPI RAG API with OpenAI embeddings, pgvector semantic similarity search, and Retrieval-Augmented Generation (RAG) workflows.

---

Overview

This project demonstrates how modern AI retrieval systems work using:

• Document chunking
• Embedding generation
• Vector similarity search
• PostgreSQL + pgvector
• Retrieval-Augmented Generation (RAG)

The API allows users to upload text documents, generate embeddings, store vectors in PostgreSQL, perform semantic search, and ask AI-powered questions based on retrieved context.

---

Features

• Upload .txt documents
• Automatic document chunking
• OpenAI embedding generation
• Vector storage using PostgreSQL + pgvector
• Semantic similarity search
• RAG-based AI question answering
• FastAPI REST API
• Swagger API documentation
• Production-style backend structure

---

Tech Stack

Technology	Purpose
Python 3.12	Backend language
FastAPI	API framework
PostgreSQL	Database
pgvector	Vector similarity search
SQLAlchemy	ORM
OpenAI API	Embeddings + LLM
Uvicorn	ASGI server

---

Project Structure

SEMANTIC-SEARCH-RAG/
│
├── app/
│   ├── chunking.py
│   ├── config.py
│   ├── database.py
│   ├── embeddings.py
│   ├── main.py
│   ├── models.py
│   ├── rag.py
│   └── schemas.py
│
├── docs/
│   └── sample.txt
│
├── .env
├── requirements.txt
└── README.md

---

Architecture

Document Upload
        ↓
Text Chunking
        ↓
Embedding Generation
        ↓
Vector Storage (pgvector)
        ↓
Semantic Similarity Search
        ↓
Relevant Context Retrieval
        ↓
LLM Response Generation

---

Installation

1. Clone Repository

git clone https://github.com/your-username/semantic-search-rag.git

cd semantic-search-rag

---

2. Create Virtual Environment

Windows

python -m venv venv

venv\Scripts\activate

Linux / macOS

python3 -m venv venv

source venv/bin/activate

---

3. Install Dependencies

pip install -r requirements.txt

---

PostgreSQL Setup

Create Database

CREATE DATABASE semantic_rag_db;

---

Enable pgvector Extension

CREATE EXTENSION IF NOT EXISTS vector;

---

Environment Variables

Create .env file in project root:

OPENAI_API_KEY=your_openai_api_key

DATABASE_URL=postgresql://postgres:password@localhost:5432/semantic_rag_db

EMBEDDING_MODEL=text-embedding-3-small

CHAT_MODEL=gpt-4.1-mini

---

Run Application

uvicorn app.main:app --reload

Server:

http://localhost:8000

Swagger API Docs:

http://localhost:8000/docs

---

API Endpoints

Upload Document

POST /upload

Uploads a .txt file, chunks text, creates embeddings, and stores vectors in PostgreSQL.

---

Semantic Search

POST /search

Example request:

{
  "query": "Can employees work remotely?",
  "limit": 3
}

---

Ask AI Question (RAG)

POST /ask

Example request:

{
  "question": "How many vacation days do employees receive?",
  "limit": 3
}

---

List Stored Chunks

GET /documents

---

Delete Stored Chunks

DELETE /documents

---

Example Semantic Search Flow

User Query
    ↓
Embedding Generation
    ↓
Vector Similarity Search
    ↓
Top-K Matching Chunks

---

Example RAG Flow

User Question
    ↓
Query Embedding
    ↓
Semantic Retrieval
    ↓
Relevant Document Chunks
    ↓
Prompt Construction
    ↓
LLM Generated Response

---

Example Queries

Can employees work from home?

How many vacation days do employees receive?

What happens if a laptop is lost?

How much can employees spend on meals?

---

Learning Goals

This project demonstrates:

• Embedding workflows
• Vector databases
• Semantic similarity search
• RAG architecture
• pgvector integration
• AI backend engineering
• FastAPI development

---

Future Improvements

• PDF support
• DOCX support
• OCR support
• Redis caching
• Hybrid search
• Metadata filtering
• Authentication
• Docker support
• Streaming responses
• AI memory system
• Conversation history
• Background workers

---

License

MIT