Go Search Engine

This project is a simple, in-memory search engine written in Go. It features TF-IDF for document ranking, a Trie data structure for efficient prefix-based searching (autocomplete), and Redis for caching search results to improve performance.

Features

• In-Memory Indexing: Builds an inverted index and a document dictionary in memory for fast lookups.
• TF-IDF Ranking: Ranks search results based on Term Frequency-Inverse Document Frequency (TF-IDF) to deliver more relevant results.
• Prefix Search with Trie: Utilizes a Trie data structure to provide fast and efficient prefix-based search suggestions.
• Redis Caching: Caches search results in Redis to reduce latency on repeated queries.
• Comprehensive Tests: Includes unit tests and benchmarks to ensure correctness and performance.

Project Structure

.
├── go.mod
├── go.sum
├── main.go
├── index/
│   ├── index.go      # Defines interfaces and data structures for indexing
│   └── inmemory.go   # In-memory implementation of the indexer
├── query/
│   ├── inmemory.go   # Implementation of the query engine with TF-IDF and caching
│   ├── query.go      # Defines the query engine interface
│   └── trie.go       # Trie data structure for prefix search
└── tests/
    ├── data.go       # Test data for documents
    └── search_test.go # Unit tests and benchmarks for the search engine

Prerequisites

Before you begin, ensure you have the following installed:

• Go (version 1.21 or later)
• Redis

Getting Started

1. Clone the repository:

   git clone https://github.com/vr-varad/search-engine.git
   cd search-engine

2. Install dependencies:

   go mod tidy

3. Ensure Redis is running: Make sure your Redis server is running on the default port (localhost:6379).

Usage

To run the application, execute the main.go file. This will initialize the indexer with sample documents, perform a sample search, and print the results.

go run main.go

You can modify the main.go file to experiment with different documents and search queries.

Running Tests

This project includes both unit tests and benchmarks.

• To run all unit tests:

  go test ./...

• To run the benchmarks:

  go test -bench=.