v1.16.3

Change log

Improvements

• #7755, #7588 - Respect search and point retrieve timeout when trying to access segments
• #7685 - Respect telemetry timeout when fetching shard statistics
• #7715 - Log snapshot download duration and speed

Bug fixes

• #7787, #7791 - Fix WAL delta transfer corrupting replica after a previous full transfer was aborted
• #7801, #7805 - Fix flush losing changes on transient disk IO errors, potentially corrupting data
• #7792 - Fix incorrectly aborting shard transfers when dropping unrelated shard
• #7741 - Fix flush error in Gridstore, potentially corrupting data when quickly alternating inserts/deletes
• #7702 - Fix flush data race in Gridstore, potentially corrupting data when storage is cleared in parallel
• #7759, #7782 - Fix handling of collection names with weird characters, breaking snapshot transfers for example
• #7788 - Fix snapshot metrics not always reporting when zero (snapshot_{creation,recovery}_running, snapshot_created_total)
• #7783 - Fix incorrectly reporting optimization errors in telemetry on panic
• #7765 - Fix slow shutdown on SIGINT when optimizations are running
• #7690 - Fix Qdrant not building on Windows ARM64
• #7683 - Keep RocksDB support until 1.18.0 in development builds

v1.16.2

Change log

Improvements

• #7607 - Improve request timeout handling for telemetry and metrics
• #7623 - Add user agent to HTTP requests sent by Qdrant server

Bug fixes

• #7674 - Fix critical WAL bug that could break consensus or cause data corruption on restart
• #7684 - Fix consensus crash when applying consensus snapshot with non-replicated collection
• #7620 - Fix panic during search on segments with empty HNSW graph
• #7629, #7640, #7673 - Fix shard resource cleanup when shard is replaced, prevent deadlock on small CPUs
• #7621, #7626 - Fix payload index storage still flushing after removal, fixing data corruption and IO errors on Windows
• #7624, #7627 - Fix Gridstore storage still flushing after wipe, fixing data corruption and IO errors
• #7614, #7618 - Fix Docker/WSL on Windows with bind mount corrupting storage
• #7678 - Fix collections_vector_total metric reporting -0.0 if there are no vectors
• #7649 - Also report collection_indexed_only_excluded_points metric if zero

v1.16.1

Change log

Improvements

• #7514, #7572 - Make batch queries up to 3 times faster on full scans by reading each point only once
• #7551 - Actively migrate vector, payload and payload index storage from RocksDB into Gridstore on startup for better and more predictable performance
• #7579 - Add 60s internal timeout for telemetry/metrics endpoints to prevent long hanging tasks
• #7557 - Add validation to restart shard transfer operation
• #7446 - Defer Gridstore flushing to make flushing behavior consistent with all other storage components
• #7580 - Improve consensus WAL compaction logging to aid debugging
• #7598 - Make timeout for inference requests user configurable

Bug fixes

• #7564 - Fix panic at startup on old clusters with user defined sharding, if not updated to Qdrant 1.15.5 first
• #7577 - Fix breaking Raft by killing node at specific time during consensus snapshot, preventing potential crash loop
• #7587 - Fix corrupting WAL with broken flush edge case after WAL is cleared or truncated
• #7570 - Fix incorrect rescoring default on mutable segments when using binary quantization
• #7569, #7575 - Spawn search/update tasks on the correct runtime, significantly reduce number of general/actix threads
• #7558 - Fix data race in shard transfers, wait on transfer to start before initializing shards
• #7556 - Fix incorrect log message when failing to read lock segment for some time

v1.16.0

Change log

Features 🌰

• https://github.com/qdrant/qdrant/milestone/33?closed=1 - Inline Storage: Add option to inline vectors in HNSW graph for efficient IO usage (docs)
• https://github.com/qdrant/qdrant/milestone/37?closed=1 - Tenant promotion mechanism for tiered multitenancy: (docs)
  • Add ReplicatePoints action to promote payload based tenant into dedicated shard key (docs)
  • Add fallback shard key for intelligent routing to tenants that are or are not promoted to a dedicated shard (docs)
  • On shard key creation, allow to specify initial state of new replicas
• #7414 - Add ACORN-1 search method, accurate search over many filtered points at the cost of performance (docs)
• #7408 - Add ASCII folding (normalization) to full text indices, fold diacritics into ASCII characters (docs)
• #7006 - Add conditional update functionality, only apply update on points matching filter (docs)
• #7100 - Add text_any full text filter to match any query term (docs)
• #7065 - Add option to customize RRF k parameter (docs)
• #7222 - In strict mode, specify maximum number of payload indices per collection
• #7123 - Add custom key-value metadata to collections (docs)
• #7291 - Add profiler to log slow point update and read requests

Improvements 🏎️

• #7385 - When loading Gridstore, populate tracker data into memory for faster first access
• #7407, #7405 - Spawn updates and flush workers task on updates runtime, use two system threads less for each local shard
• #7413 - Use system thread on demand in WAL, use one less system thread per local shard by default
• #7468 - Use atomic bit flags on HNSW construction to significantly increase indexing performance
• #7052, #7471 - Implement AVX512 SIMD optimizations for binary quantization on modern x86_64 CPUs
• #7433 - Switch new mutable payload indices and storage from RocksDB to Gridstore for better performance
• #7508 - Enable quantization in appendable segments by default, improving search performance
• #7347 - Change default score of query-less prefetch to 1.0 to ease score boosting
• #7369 - Don't explicitly disable strict mode by default
• #7345 - Simplify internal handling of copy-on-write segments, now write incoming updates to dedicated segments directly
• #7293, #7523 - Add warnings field to collection info, report misconfiguration
• #7319, #7401 - Report more helpful error messages on file IO errors
• #7377 - When using inference, propagate rate limit responses
• #7434 - Rate limit slow request warning to prevent spamming logs
• #7373 - Log shard transfers as a result of the consensus recovery procedure
• #7337 - Tone down optimizer logging to make it less verbose
• #7370 - Add TARGET_CPU and JEMALLOC_SYS_WITH_LG_PAGE build parameters to Docker image

Metrics 📈

• #7302, #7441 - In metrics, report point and vector counts per collection and vector name (collection_{points,vectors})
• #7307 - In metrics, report number of points skipped in indexed_only search (collection_indexed_only_excluded_points)
• #7301 - In metrics, report global effective minimum and maximum shard replication count (collection_active_replicas_{min,max})
• #7310, #7516 - In metrics, report total number of non-active replicas (collection_dead_replicas)
• #7316, #7480 - In metrics, expose collection_running_optimizations with number of optimizers running per collection
• #7497 - In metrics, report running, created and recovered number of snapshots (snapshot_{creation,recovery}_running, snapshot_created_total)
• #7484 - In metrics, report active thread count (process_threads)
• #7451 - In metrics, report number of open file descriptors and the limit (process_{open,max}_fds)
• #7487 - In metrics, report open mmaps and system limit (process_open_mmaps, system_max_mmaps)
• #7482 - In metrics, report total number of major and minor page faults (process_{minor,major}_page_faults_total)
• #7438 - In metrics, add configuration option to prefix all metrics with qdrant_ or something else

Bug fixes 🕵️

• #7527 - Fix logger API allowing arbitrary file writes, now this can only be configured through configuration
• #7530 - Abort pending search tasks when search is cancelled, fixing optimizer instability under huge load
• #7533 - Abort other blocking tasks such as retrieve and snapshot prematurely if the caller gets cancelled
• #7531 - Cancel ongoing searches more aggressively if the search is cancelled
• #7517 - Fix resharding down panic if no shard key is provided on collection with custom sharding
• #7469 - Fix panic on certain queries with unknown vector name
• #7375 - Forbid peer to join cluster with URI that is already used, which could break a cluster
• #7400 - Fix corrupt segments on load if segment was partially flushed, prevent payload index corruption
• #7342, #7404, #7427 - Force flush all segments when taking snapshot to prevent data corruption
• #7381 - Fix flush ordering on segments currently being snapshotted, fixing data consistency on crash
• #7388, #7416 - Fix flush ordering with concurrent flushes to ensure data consistency
• #7512, #7521 - Fix dummy shards not allowing snapshot recovery

Web UI 🫂

• https://github.com/qdrant/qdrant-web-ui/releases/tag/v0.2.0 - New web UI design to match Qdrant Cloud

Deprecations 🚧

• #7454 - Remove init_from collection API, deprecated since Qdrant 1.15
• #7449 - Remove lock API, deprecated since Qdrant 1.15
• #7047 - Remove old internal shard key format, deprecated and migrated away from in Qdrant 1.15
• #7450 - Remove payload filter from RBAC/JWT, deprecated since Qdrant 1.15, API keys using it are rejected
• #7183 - Deprecate old variant of vector output

v1.15.5

Change log

Improvements

• #7157 - Acknowledge update/delete by filter operations on flush, preventing very slow restart
• #7217, #7218, #7219, #7220, #7221 - Add API validation to min_should, filters, point update batch and others
• #7235 - Add timeout parameter to remove peer operation
• #7320 - Decrease internal update batch sizes to minimize search latency spikes on large user batches
• #7233 - Limit number of segments loaded in parallel, preventing potential OOM on large clusters
• #7222 - Add strict mode configuration to specify max number of payload indices
• #7240 - Improve error reporting on mutable ID tracker load errors
• #7151 - Improve error reporting on flush problems
• #7244 - Remove vector count field from collection info
• #7177 - Do not anonymize peer ID in telemetry data

Bug fixes

• #7263 - Fix not flushing mutable ID tracker files after creation, potentially causing segment corruption
• #7248 - Fix data race at the end of snapshot creation causing missing points
• #7298, #7306 - Fix data race during snapshots corrupting point data if a point is moved
• #7241, #7265, #7267, #7269, #7277 - Fix potential deadlock in REST runtime while streaming shard snapshot
• #7015 - Fix potential deadlock on REST server runtime
• #7172 - Fix potential recursive deadlock when fetching all vectors
• #7203 - Fix potential deadlock in scroll API
• #7303 - Fix incorrectly deleting old point versions from segments
• #7194 - Fix upsert operations with duplicate point IDs not being applied properly
• #7252 - Fix phrase matching ignoring unknown tokens
• #7264 - Fix strict mode validation on nested filters

v1.15.4

Change log

Improvements

• #7005 - Reduce Docker image size by 10-40%
• #7073 - Adjust metrics histogram buckets, show empty buckets and remove small ones
• #7111 - Include SBOM in Docker image
• #7119 - Sign Docker images with cosign
• #7110 - Actively migrate away from old shard key format on disk to a more robust format
• #7120 - Measure segment size on disk more reliably for improved available disk space checks

Bug fixes

• #7133 - Fix not loading some full text index types correctly, temporarily resulting in an empty index
• #7109 - Fix incorrectly reusing old deleted points in proxy segments
• #7106 - Preserve segment ID mapping when proxying all segments for a snapshot, potentially mixing up segments

v1.15.3

Change log

Improvements

• #7002 - Optimize dot product calculation on AVX systems
• #7049 - In Nix package, use rustup for Rust version management

Bug fixes

• #7056 - Fix local BM25 not working as expected due to default parameter differences

v1.15.2

Change log

Improvements

• #6891 - Implement BM25 inference in Qdrant locally
• #6926 - Improve performance of mutable map index, used for full text, integers and more
• #6993 - Make log buffer size adjustable when logging to a file

Bug fixes

• #6954 - Fix consistency problem in null index storage by deferring writes until flush
• #6977 - Fix consistency problem in boolean index storage by deferring writes until flush
• #6994 - Fix consistency problem for deleted vectors by deferring writes until flush
• #6966 - Fix flush logic in memory mapped bit flags structure, improving data consistency
• #6975 - Delete versionless points before WAL replay, fix incorrect use of corrupt points
• #6952 - Fix date time error in custom score boosting formula when having multiple nodes
• #6998 - In score boosting formula validation error, show each index type only once
• #6959 - Fix range bounds not being inclusive for linear decay function
• #6958 - Fix default storage path and missing web UI assets in Debian package

Web UI

• qdrant/qdrant-web-ui#299 - Show shard distribution of collection in cluster as replica/node matrix

v1.15.1

Changelog

Improvements

• #6931 - gRPC HealthCheck method now works without authentication, making it consistent with REST methods.
• #6923 - Use populate storage components with MADV_SEQUENTIAL for better IO performance on indexing

Bug Fixes

• #6932 - Fix point shard routing broken in 1.15 (Points created in 1.15.0 in multi-shard collections might not behave properly on updates!)
• #6903 - Fix compile error in MUSL builds
• #6910 - Fix panic in /metrics on empty collection data
• #6916 - Fix UUID index storage, resulted in missing filter results.

v1.15.0

Change log

Features 📜

• milestone/25 - Phrase Matching in Full-Text index [Docs]
• milestone/28 - Stop words support in Full-Text index [Docs]
• milestone/29 - New multilingual tokenizer enabled by default [Docs]
• milestone/32 - Snowball Stemmer in Full-Text index, [Docs]
• milestone/30 - Asymmetric binary quantization [Docs]
• milestone/31 - 2-bit and 1.5-bit Binary Quantization encoding [Docs]
• milestone/26 - Maximum Marginal Relevance [Docs]
• #6682 - Verify file system compatibility on process start

Improvements 🪨

• milestone/16 - Migrate away from RocksDB
• milestone/27 - HNSW Healing on optimization
• #6479 - Speedup additional links construction by measuring sub-graph connectivity
• #6563, #6601 - Inference usage now reported in API response
• #6611 - Added major and minor tags for Docker builds
• #6685 - Speed up mutable posting list for upsertions
• #6729 - Reuse quantized vectors during HNSW index building
• #6713 - Optimize Gridstore insertions
• #6750 - Use sequential reads when iterating over Gridstore
• #6757 - Enable pod role based auth for S3 snapshots
• #6725 - Improve read request handling during snapshot recovery
• #6785 - Avoid re-optimizations when loading a mmap-backed immutable text index
• #6778, #6817 - Properly handle inconsistencies in consensus during custom shards creation
• #6779 - Avoid IO during evaluating HNSW index for re-optimization
• #6796 - Use sequential reads for building quantized vectors
• #6794 - Avoid double condition check during filtering
• #6804 - Optimize has_id condition
• #6844 - Adjust default values for better performance

Bug fixes 🫙

• #6593 - Use ID tracker to estimate internal points range
• #6614 - Improve Gridstore wipe/clear functionality
• #6632 - Fix default replication factor
• #6640 - Fix significant memory usage in Distance Matrix API on high sample size
• #6739 - Fix IDF computation with deleted points
• #6755 - Prevent shards from getting stuck in Initializing state
• #6759 - Properly propagate inference errors
• #6787 - Add flag to break optimization loop on panic
• #6862 - Avoid blocking async runtime when loading local shard
• #6800 - Fix bug that causes all replicas to die if node restarted during resharding
• #6882 - Fix broken is_empty filter on new points

Web-UI 🎨

• https://github.com/qdrant/qdrant-web-ui/releases/tag/v0.1.41 - New "Create Collection" form, Simplify JWT form, other fixes

Deprecations 🚧

• init_from parameter of the collection creation API is deprecated and going to be removed in v1.16

  • Reason: init_from creates false expectation of data consistency and reliability of this operation.
  • Alternative: prefer usage of migration tool or snapshots, read more in the tutorial

• Payload based filters in JWT going to be removed in v1.16

  • Reason: Payload based filters are inconsistent for read/write operations. Many update operations are not implemented due to security risks and complexity. Lack of real-life usage.
  • Alternative: prefer collection-based access control.

• mmap_threshold parameter of the collection optimizer is deprecated and going to be removed in v1.16

  • Reason: starting from v1.12, all vector storages exclusively use memory maps, explicit usage of mmap_threshold is obsolete
  • Alternative: control memory cache behaviour of Qdrant storages with on_disk: true/false parameter

• Lock API is deprecated and going to be removed in v1.16

  • Reason: lack of real-life usage
  • Alternative: JWT tokens can be configured to provide read-only access to Qdrant

• #6760 - Removed max_optimization_threads from config