🔍 Search Engine

A modern and scalable search engine built with FastAPI and RedisSearch, allowing you to create dynamic schemas and index documents for real-time search.

📖 Introduction

This is a simple yet powerful search engine solution designed for developers and small to medium-sized projects that need fast, full-text search capabilities without the complexity of enterprise search solutions.

Purpose

The Search Engine provides a straightforward way to implement search functionality in your applications. Built with Python and RedisSearch, it offers:

• Simplicity: Easy to set up and use - just run Docker Compose and you're ready to go
• Flexibility: Create custom search schemas dynamically without database migrations
• Performance: Leverages RedisSearch for lightning-fast search operations
• Developer-Friendly: RESTful API with automatic interactive documentation

Who Can Benefit

This solution is ideal for:

• Developers building applications that need search functionality (e-commerce, content management, document systems)
• Small to Medium Projects that require search capabilities without the overhead of complex search infrastructure
• Prototyping and MVP development where you need search features quickly
• Learning Projects to understand how search engines work under the hood
• Startups and small teams that need a cost-effective search solution

Why This Solution?

While there are more complex search solutions available (Elasticsearch, Solr, etc.), this engine focuses on simplicity and ease of use. If you need:

• Quick setup and deployment
• Simple API integration
• Full-text search with basic features
• A lightweight solution that doesn't require extensive configuration

Then this search engine is perfect for you. It's built with modern Python technologies (FastAPI) and RedisSearch, making it both performant and easy to maintain.

📋 Table of Contents

• Introduction
• About the Project
• Features
• Technologies
• Prerequisites
• Installation
• Configuration
• Usage
• API Reference
• Project Structure
• Examples
• Development

🎯 About the Project

This project is a search engine that uses RedisSearch to provide full-text search capabilities on documents. It allows you to create dynamic schemas with different field types (text, numeric, tag) and index documents for fast and efficient searching.

✨ Features

• ✅ Dynamic search schema creation
• ✅ Document indexing with field validation
• ✅ Full-text search with pagination
• ✅ Support for multiple field types (text, numeric, tag)
• ✅ Sortable fields and customizable weights
• ✅ Automatically documented RESTful API
• ✅ Clean architecture with separation of concerns
• ✅ Consistent error handling

🛠 Technologies

• FastAPI - Modern and fast web framework for APIs
• RedisSearch - Full-text search engine based on Redis
• Redis - In-memory database
• Pydantic - Data validation
• Docker - Containerization
• Python 3 - Programming language

📦 Prerequisites

• Docker and Docker Compose installed
• Git

🚀 Installation

1. Clone the repository:

git clone https://github.com/FabioAugustoRodrigues/search-engine.git
cd search-engine

2. Start the services:

docker compose up -d

3. The API will be available at http://localhost:8400

4. Access the interactive documentation at:

   • Swagger UI: http://localhost:8400/docs
   • ReDoc: http://localhost:8400/redoc

That's it! The project is ready to use. All dependencies and services (FastAPI application and Redis with RedisSearch) are automatically configured and started via Docker Compose.

⚙️ Configuration

The project is pre-configured to work out of the box. The Redis connection is automatically set up between the containers.

If you need to customize the Redis connection URL, you can create a .env file in the project root:

REDIS_URL=redis://redis:6379

Note: When using Docker Compose, the default configuration works without any additional setup.

📖 Usage

1. Create a Schema

First, you need to create a schema that defines the structure of the documents that will be indexed.

Endpoint: POST /api/v1/schemas

Request Example:

{
  "name": "products",
  "fields": [
    {
      "name": "name",
      "type": "text",
      "sortable": true,
      "weight": 5.0
    },
    {
      "name": "price",
      "type": "numeric",
      "sortable": true
    },
    {
      "name": "category",
      "type": "tag",
      "sortable": false
    }
  ]
}

Field Types:

• text: Text field for full-text search
• numeric: Numeric field for comparisons and sorting
• tag: Tag field for categorical filtering

Field Parameters:

• name: Field name (required)
• type: Field type - text, numeric, or tag (required)
• sortable: Whether the field can be used for sorting (default: false)
• weight: Field weight in search (only for text, default: 1.0)

2. Index a Document

After creating the schema, you can index documents.

Endpoint: POST /api/v1/documents

Request Example:

{
  "schema_name": "products",
  "fields": {
    "name": "Dell XPS 13 Laptop",
    "price": 1299.99,
    "category": "electronics"
  }
}

Important: The document fields must exactly match the fields defined in the schema.

3. Search Documents

Search documents using free text.

Endpoint: GET /api/v1/documents

Query Parameters:

• schema_name (required): Schema name
• text (required): Search text
• limit (optional): Maximum number of results (default: 10)
• offset (optional): Number of results to skip (default: 0)

Example:

GET /api/v1/documents?schema_name=products&text=laptop&limit=10&offset=0

Response:

{
  "details": {
    "message": "Documents retrieved successfully",
    "documents": {
      "total": 1,
      "documents": [
        {
          "id": "document-uuid",
          "fields": {
            "name": "Dell XPS 13 Laptop",
            "price": 1299.99,
            "category": "electronics"
          },
          "score": 1.5
        }
      ]
    }
  },
  "status_code": 200
}

4. List Schemas

List all created schemas.

Endpoint: GET /api/v1/schemas

📚 API Reference

Schemas

POST /api/v1/schemas

Creates a new search schema.

Request Body:

{
  "name": "string (alphanumeric/underscore)",
  "fields": [
    {
      "name": "string",
      "type": "text | numeric | tag",
      "sortable": boolean,
      "weight": number (only for text)
    }
  ]
}

Responses:

• 200: Schema created successfully
• 409: Schema already exists
• 500: Error creating schema

GET /api/v1/schemas

Lists all created schemas.

Responses:

• 200: List of schemas

Documents

POST /api/v1/documents

Indexes a new document.

Request Body:

{
  "schema_name": "string",
  "fields": {
    "field1": "value1",
    "field2": "value2"
  }
}

Responses:

• 200: Document indexed successfully
• 400: Invalid fields (extra or missing fields)
• 404: Schema not found

GET /api/v1/documents

Searches documents by text.

Query Parameters:

• schema_name (string, required)
• text (string, required)
• limit (integer, optional, default: 10)
• offset (integer, optional, default: 0)

Responses:

• 200: Documents found
• 404: Schema not found

📁 Project Structure

search-engine/
├── app/
│   ├── api/
│   │   └── v1/
│   │       ├── document.py      # Document routes
│   │       └── schema.py        # Schema routes
│   ├── core/
│   │   ├── clients.py           # Redis client
│   │   ├── dependencies.py      # Dependency injection
│   │   ├── responses.py         # Standardized responses
│   │   └── settings.py          # Settings
│   ├── domain/
│   │   ├── exceptions/
│   │   │   └── domain_exception.py  # Domain exceptions
│   │   └── services/
│   │       ├── document_service.py  # Business logic - documents
│   │       └── schema_service.py   # Business logic - schemas
│   ├── infrastructure/
│   │   └── providers/
│   │       ├── redis_provider.py        # Basic Redis provider
│   │       └── redis_search_provider.py # RedisSearch provider
│   ├── schemas/
│   │   ├── document.py          # Pydantic models - documents
│   │   └── schema.py            # Pydantic models - schemas
│   └── validators/
│       └── fields_validator.py   # Field validator
├── docker/
│   └── Dockerfile.Python.Dev    # Development Dockerfile
├── compose.yml                   # Docker Compose
├── main.py                       # Application entry point
└── README.md                     # Documentation

Architecture

The project follows a layered architecture:

• API Layer (app/api/): REST endpoints and routes
• Domain Layer (app/domain/): Business logic and exceptions
• Infrastructure Layer (app/infrastructure/): Provider implementations (Redis, RedisSearch)
• Core (app/core/): Settings, dependencies, and shared utilities

💡 Examples

Complete Example: E-commerce Products

1. Create Schema:

curl -X POST "http://localhost:8400/api/v1/schemas" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "products",
    "fields": [
      {"name": "title", "type": "text", "sortable": true, "weight": 10.0},
      {"name": "description", "type": "text", "sortable": false, "weight": 2.0},
      {"name": "price", "type": "numeric", "sortable": true},
      {"name": "category", "type": "tag", "sortable": true},
      {"name": "brand", "type": "tag", "sortable": false}
    ]
  }'

2. Index Products:

curl -X POST "http://localhost:8400/api/v1/documents" \
  -H "Content-Type: application/json" \
  -d '{
    "schema_name": "products",
    "fields": {
      "title": "Samsung Galaxy S23 Smartphone",
      "description": "Android smartphone with 256GB storage",
      "price": 899.99,
      "category": "electronics",
      "brand": "samsung"
    }
  }'

3. Search Products:

curl "http://localhost:8400/api/v1/documents?schema_name=products&text=samsung&limit=5"

Example: Article System

1. Create Schema:

{
  "name": "articles",
  "fields": [
    {"name": "title", "type": "text", "sortable": true, "weight": 8.0},
    {"name": "content", "type": "text", "sortable": false, "weight": 1.0},
    {"name": "author", "type": "tag", "sortable": true},
    {"name": "published_date", "type": "numeric", "sortable": true}
  ]
}

2. Index Article:

{
  "schema_name": "articles",
  "fields": {
    "title": "Introduction to FastAPI",
    "content": "FastAPI is a modern framework for building APIs...",
    "author": "john_doe",
    "published_date": 20240115
  }
}

🔧 Development

Code Structure

• Clean Architecture: Clear separation between layers
• Dependency Injection: Use of FastAPI dependencies
• Domain-Driven Design: Domain services encapsulate business logic
• Provider Pattern: Infrastructure abstraction through providers

Contributing

1. Fork the project
2. Create a feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

📝 License

This project is licensed under the MIT License. See the LICENSE file for more details.

👤 Author

Fabio Augusto Rodrigues - @FabioAugustoRodrigues

🙏 Acknowledgments

• FastAPI for the excellent documentation
• RedisSearch for the powerful search engine
• Open source community