v1.20.0

Change Log

Highlights 🔦

Qdrant 1.20 🎃

BM25 is now supported through TextQuery: set scoring on a text index and rank points by how well that field matches the query text. Local mode supports it only for text indexes with the default tokenization. Formulas get the max, min, and acosh expressions.

The new MatchSubstring condition finds keyword values that contain a given string, using a keyword index. Payload indexes can be nested, indexing every object of an array as its own entry: with nested: offers, a text index on offers[].description matches and BM25-scores each offer separately.

Performance 🐺

With pydantic v2, parsing REST responses with vectors is about 2x quicker, and about 3x with the orjson extra (pip install 'qdrant-client[orjson]'). On protobuf 6.30 and later, gRPC writes with payloads are up to about 3x quicker.

PointStruct, PointVectors, and Batch are much quicker to build from numpy vectors, and parallel uploads of numpy arrays over REST barely add to the main process's memory.

Behavior Changes 🎭

• On macOS, parallel uploads and parallel local inference start workers with spawn instead of forkserver, which crashed on Python 3.15. Pass method="forkserver" to upload_points or upload_collection to keep the old start method (#1551, #1568).
• REST path params are percent-encoded, so names with /, #, %, or ? address the right collection, and names that are exactly . or .. raise ValueError (#1291).
• Uploads raise ValueError when ids or payload is shorter than the vectors, when parallel is negative, or when auth_token_provider is async. parallel=0 uses all CPU cores (#1578, #1531, #1520).

Features 🍬

• #1610 - support Qdrant v1.20: BM25 text queries (TextQuery) with scoring on text indexes, MatchSubstring, max, min, and acosh formula expressions, nested payload indexes, id_tracker collection config, turbo8 and turbo16 datatypes, and the code tokenizer; migrate keeps the payload and id_tracker collection params by @joein
• #1565, #1566 - skip re-validation of vectors and payloads when parsing REST responses on pydantic v2, and add an orjson extra for faster decoding by @joein
• #1513, #1514, #1562 - speed up gRPC payload conversion in both directions, especially on protobuf >= 6.30, send numpy vectors as raw bytes in gRPC uploads, and skip float lists when looking for inference objects by @joein
• #1561 - convert numpy vectors to lists before validation in PointStruct, PointVectors, and Batch by @joein
• #1558 - bump fastembed to 0.9.0 by @joein
• #1555 - cut main process memory of parallel uploads by @joein
• #1519 - add with_ to get_optimizations to request queued, completed, and idle_segments, and request completed when completed_limit is set by @joein

Fixes 🚑

• #1605, #1617 - default the text index tokenizer to word in create_payload_index, and convert WriteOrdering in create_payload_index and delete_payload_index over gRPC by @feiiiiii5, @astroknot-sheep, @joein
• #1602 - return None instead of {} for absent collection metadata over gRPC by @feiiiiii5, @joein
• #1590 - remove broken request_sync from ApiClient and AsyncApiClient by @joein
• #1553, #1588 - reset embedder state after a failed or abandoned embed_models_batch, and pass local_inference_batch_size to query embedding in local inference by @moneebullah25, @RAMZI0TO99, @joein
• #1545, #1578 - reject Batch columns of different lengths in local mode and over gRPC, and ids or payload shorter than vectors in upload_collection by @fhgffy, @joein
• #1572 - keep empty points lists in batch payload and vector updates over gRPC instead of falling back to filter by @joein
• #1490, #1563, #1570 - copy collection metadata in migrate, validate its batch_size before touching the destination, and drop its own retry loop, since upload_points already retries by @nicoloangileri, @shayanxcyber, @09Catho, @joein
• #1551, #1568 - use spawn instead of forkserver for parallel uploads and local inference on macOS by @joein
• #1549 - raise on named vectors of unsupported types over gRPC instead of uploading points without them by @joein
• #1534 - do not write the shard key into the caller's points selector by @feiiiiii5, @joein
• #1529, #1531 - count max_retries in uploads as retries after the first attempt, so the default 3 makes up to 4 attempts, use all cores for parallel=0, and reject negative values of both by @feiiiiii5, @joein
• #1509, #1520 - pass auth_token_provider and grpc_compression to gRPC uploads, and reject async token providers in uploads by @breken-ai, @nandanadileep, @joein
• #1487 - terminate interrupted parallel workers and fail fast when a worker dies by @joein
• #1473 - encode non-JSON payload values (UUID, Decimal, sets, bytes, and so on) over gRPC like REST does by @IncognitoQuack, @joein
• #1291 - percent-encode path params in the REST client and reject . and .. by @vaibzde, @vaibhavaol, @joein

Local Mode 🐈‍⬛

• #1515, #1622 - persist writes before applying them in memory, convert collection metadata to JSON values like the server, and replace meta.json atomically by @adityaanikam, @joein
• #1610 - support TextQuery (BM25 with the default tokenization only), MatchSubstring, and the new formula expressions; validate vector names and sizes like the server, reject unknown vector names in update_collection, and raise on payload index operations for a missing collection by @joein
• #1525, #1597, #1599 - accept lin_decay midpoints 0.0 and 1.0, reject invalid slice conditions, and ignore extra fields on geo points in formula queries by @aniketkrs, @digit50, @kabishou11, @joein
• #1476, #1547, #1585 - reject mismatched dense and sparse vectors, sparse indices outside the uint32 range, nested dense vectors, and payloads that can't be converted to JSON before writing by @Taranum01, @ktz03, @digit50, @joein
• #1580 - make the order of tied formula scores and array group_by values independent of the hash seed by @pacocartones, @joein
• #1485, #1573 - keep alias batches atomic on failure and reject aliases that clash with collection names, renames included by @Harsh23Kashyap, @joein
• #1500, #1542, #1560 - in grouped queries, return the lookup payload by default, resolve point ids and lookup_from in prefetches, and honor search_params (IDF corpus) by @galleonli, @nandanadileep, @j45856021-dev, @joein
• #1483, #1556 - match with_payload keys as JSON paths, including quoted keys, and treat an empty with_payload list as selecting no fields by @Harsh23Kashyap, @feiiiiii5, @joein
• #1521, #1533 - isolate collection configs and metadata from caller objects and get_collection results, and keep unset fields on partial sparse vector config updates by @lllleolin-max, @wcsnbilil, @joein
• #1532 - match server validation of score_threshold, prefetch, and fusion queries by @kabishou11, @joein
• #1530 - remove keys set to null when merging payload and collection metadata by @Yuilona, @joein
• #1474, #1523 - unwrap arrays selected by [] and [i] one level in payload paths and group_by, like the server by @IncognitoQuack, @joein
• #1493, #1512 - honor update_mode in batch_update_points and on deleted points, and reject empty selectors and empty point lists in update operations by @truecallerabreham, @xvimelese-ctrl, @Harsh23Kashyap, @joein
• #1505, #1508 - allow deleting sparse vectors next to an unnamed dense vector, align create_vector_name and delete_vector_name with the server, and reject an empty sparse vector name in create_collection by @Marcuswang0824, @joein
• #1497, #1503 - validate collection names like the server, keep collection paths inside the storage folder, and return False when deleting an absent collection by @Harsh23Kashyap, @joein
• #1498 - remove migration of legacy storage.dbm storage (qdrant-client < 1.1.7) by @joein
• #1489 - skip non-object array elements in nested filters by @winklemad, @joein
• #1481 - treat all spellings of a UUID as one point id by @tjwjdrok, @joein

Thanks to everyone who contributed to the current release!
@feiiiiii5 @astroknot-sheep @moneebullah25 @RAMZI0TO99 @fhgffy @nicoloangileri @shayanxcyber @09Catho @breken-ai @nandanadileep @IncognitoQuack @vaibzde @vaibhavaol @adityaanikam @aniketkrs @digit50 @kabishou11 @Taranum01 @ktz03 @pacocartones @Harsh23Kashyap @galleonli @j45856021-dev @lllleolin-max @wcsnbilil @Yuilona @truecallerabreham @xvimelese-ctrl @Marcuswang0824 @winklemad @tjwjdrok @joein