Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions vector-search-memory/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Apache Answer Plugin: In-Memory Vector Search

A lightweight, pure-Go, in-memory vector search plugin for [Apache Answer](https://github.com/apache/answer). Designed for **development and testing** -- no external services, no Docker, no CGo required.

## When to Use This Plugin

- **Development** -- quickly test semantic search without setting up a database
- **Testing** -- verify vector search integration in CI/CD pipelines
- **Small demos** -- run Answer with vector search on a single machine with zero setup

**Not recommended for production** -- all data is stored in memory and lost on restart.

## Installation

Build Apache Answer with this plugin:

```bash
./answer build --with github.com/apache/answer-plugins/vector-search-memory
```

No special environment variables needed. Works with `CGO_ENABLED=0`.

## Configuration

After enabling the plugin in the Admin UI (**Admin > Plugins > Vector Search**), configure the following fields:

| Field | Description | Example |
|---|---|---|
| **Embedding API Host** | OpenAI-compatible API base URL | `https://api.openai.com` |
| **Embedding API Key** | API key for the embedding service | `sk-...` |
| **Embedding Model** | Model name for generating embeddings | `text-embedding-3-small` |
| **Embedding Level** | `question` embeds question + all answers + comments together; `answer` embeds each answer separately | `question` |
| **Similarity Threshold** | Minimum cosine similarity score (0-1). Default `0` means no filtering | `0.5` |

No connection endpoint or database path is needed -- everything runs in-process.

## How It Works

- Stores all document embeddings in a Go `map[string]*document` guarded by `sync.RWMutex`
- Search performs brute-force cosine similarity over all stored vectors
- Embedding dimensions are auto-detected from the configured model
- Changing the embedding model clears all stored documents (since dimensions may differ)
- A full sync of all questions/answers is triggered when the plugin starts

## Limitations

- **No persistence** -- data is lost when Answer restarts
- **No scalability** -- brute-force search is O(n) per query; suitable for thousands of documents, not millions
- **Memory usage** -- each document stores a full embedding vector in RAM

For production use, consider [pgvector](../vector-search-pgvector/), [Elasticsearch](../vector-search-elasticsearch/), [Weaviate](../vector-search-weaviate/), [Milvus](../vector-search-milvus/), [Qdrant](../vector-search-qdrant/), or [ChromaDB](../vector-search-chromadb/).

## License

[Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0)
55 changes: 55 additions & 0 deletions vector-search-memory/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
module github.com/apache/answer-plugins/vector-search-memory

go 1.23.0

require (
github.com/apache/answer v1.7.0
github.com/apache/answer-plugins/util v1.0.3-0.20250107030257-cf94ebc70954
github.com/segmentfault/pacman v1.0.5-0.20230822083413-c0075a2d401f
)

require (
github.com/LinkinStars/go-i18n/v2 v2.2.2 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/bytedance/sonic v1.12.2 // indirect
github.com/bytedance/sonic/loader v0.2.0 // indirect
github.com/cloudwego/base64x v0.1.4 // indirect
github.com/cloudwego/iasm v0.2.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.5 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect
github.com/gin-gonic/gin v1.10.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.22.1 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/wire v0.5.0 // indirect
github.com/gorilla/css v1.0.1 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.8 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/microcosm-cc/bluemonday v1.0.27 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/segmentfault/pacman/contrib/i18n v0.0.0-20230822083413-c0075a2d401f // indirect
github.com/syndtr/goleveldb v1.0.0 // indirect
github.com/tidwall/gjson v1.17.3 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.12 // indirect
github.com/yuin/goldmark v1.7.4 // indirect
golang.org/x/arch v0.10.0 // indirect
golang.org/x/crypto v0.36.0 // indirect
golang.org/x/net v0.38.0 // indirect
golang.org/x/sys v0.31.0 // indirect
golang.org/x/text v0.23.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
xorm.io/builder v0.3.13 // indirect
xorm.io/xorm v1.3.2 // indirect
)
Loading