Skip to content

bench: add qdrant comparison to vector benchmark#144

Merged
kacy merged 2 commits intomainfrom
bench/qdrant-comparison
Feb 15, 2026
Merged

bench: add qdrant comparison to vector benchmark#144
kacy merged 2 commits intomainfrom
bench/qdrant-comparison

Conversation

@kacy
Copy link
Copy Markdown
Owner

@kacy kacy commented Feb 15, 2026

summary

  • adds qdrant as an optional fourth comparison target in the vector similarity benchmark
  • opt-in via --qdrant flag so existing usage is unchanged
  • uses official qdrant/qdrant docker image and qdrant-client python SDK
  • matches HNSW parameters (M=16, ef_construct=64, cosine) across all systems
  • compatible with the ember-grpc benchmark added in main

what was tested

  • cargo fmt and cargo clippy pass clean
  • ran full 100k vector benchmark locally across ember, qdrant, and pgvector

local results (100k vectors, 128-dim, cosine, k=10)

metric ember pgvector qdrant
insert (vectors/sec) 1,382 1,239 5,558
query (queries/sec) 1,776 519 337
query p99 (ms) 0.82 4.85 10.31
memory (MB) 29 193 31

ember query throughput is 5.3x faster than qdrant and 3.4x faster than pgvector at this scale. qdrant leads on insert throughput thanks to native batch upsert.

chromadb skipped — pydantic v1 is broken on python 3.14.

design considerations

  • qdrant is opt-in (--qdrant flag) rather than always-on to keep docker optional for the default run
  • --ember-only --qdrant works as a 2-system comparison (no chromadb/pgvector needed)
  • uses query_points API (current qdrant SDK; search was removed)
  • qdrant uses integer point IDs with string names in payload, since it doesn't support arbitrary string IDs

kacy added 2 commits February 14, 2026 19:22
adds qdrant as an optional fourth comparison target in the vector
similarity benchmark suite. opt-in via --qdrant flag to avoid
requiring docker for the default run.

- qdrant python client class using official SDK
- docker container lifecycle (startup, health check, teardown)
- results parsing, display table, and CSV output columns
- pip dependency management for qdrant-client
combines qdrant comparison (this branch) with ember-grpc benchmark
(from main). both systems now available in choices, flags, and CSV
output.
@kacy kacy merged commit d704dfa into main Feb 15, 2026
6 of 7 checks passed
@kacy kacy deleted the bench/qdrant-comparison branch February 15, 2026 05:22
kacy added a commit that referenced this pull request Feb 19, 2026
adds qdrant as an optional fourth comparison target in the vector
similarity benchmark suite. opt-in via --qdrant flag to avoid
requiring docker for the default run.

- qdrant python client class using official SDK
- docker container lifecycle (startup, health check, teardown)
- results parsing, display table, and CSV output columns
- pip dependency management for qdrant-client
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant