GG-49932 Vector query fast paths: one-pass response decode, shared ctypes classes, sane paging default - #84
Closed
PakhomovAlexander wants to merge 3 commits into
Closed
GG-49932 Vector query fast paths: one-pass response decode, shared ctypes classes, sane paging default#84PakhomovAlexander wants to merge 3 commits into
PakhomovAlexander wants to merge 3 commits into
Conversation
Contributor
Author
|
Pre-review gate: the private EE suite (ggprivate-python-thin-client, branch gg-50205 — the GG-49286 e2e matrix plus the older vector tests) passes 53/53 with this branch installed as the client, against a live UE node built from CE/EE master + gg-50942 + gg-50943 ( |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket: GG-49932
Vector-query response deserialization was the bulk of this client's per-query cost. Profiling on a
settled 100k x 1536-d index (server = CE/EE master + gg-50942 + gg-50943, the branches that merge
next) put ~24% of client CPU in dynamic ctypes class creation - one
type()call per element perrow - and the rest in the triple walk: page parse,
to_python, then a second full parse of everywrapped binary value in
unwrap_binary.Three commits, layered so each is measurable on its own:
1. Cache ctypes classes per shape;
page_sizedefaults tok; reject non-positivek.cached_c_typegeneralizes the GG-50683 memo to the six sites that still built a class per parse:wrapped payloads, binary objects and their schema elements, primitive arrays, map/collection
containers, struct arrays, response envelopes. The classes are only ever read, never mutated, so
one shared class per shape is safe; shapes recur because ctypes caches array types and the leaf
classes come from the same cache. The
page_size=1default meant one server round trip per resultrow (20 recv calls per k=10 query); a vector query returns at most
krows, so the default is nowone page holding the whole result.
k < 1raisesValueErrorbefore any I/O, mirroring theserver-side bound from GG-49668 (the 10000 ceiling stays server-side - it is configurable there).
2. Decode vector query responses in one pass (
VectorResponse).The same pattern as the existing
SQLResponse: rows leaveparse()as final Python values, readstraight off the response buffer - no per-element ctypes classes, no payload blob copy, no second
parse. Direct readers cover what vector rows actually carry (long/int/double/string keys, binary
objects with primitive or float-array fields, raw float scores); any other element falls back to
the generic machinery at the same stream position, one element at a time. Covers the flagged and
the legacy (v1 map) row layouts, sync and asyncio. The vector cursors stop calling
unwrap_binary.3. Server-free tests: the
kguard, thepage_sizedefault, and the shared-class cache.Measured (GCP VM, one bracketed invocation: base -> 1 -> 1+2 -> base again)
2000 queries per leg after warm-up, ef=40, dbpedia-openai 100k, single-segment index, drift
bracket 0.6-5%. Mean ms per query (QPS) / client CPU ms:
(key, value)k=10(key, value, score)(key, value)k=100page_size=1), k=10The baseline also retained 562-5422 objects per query (the per-row classes awaiting gc); both
optimized arms retain none.
Correctness
{k=10, k=10 with page_size=3 (paging), k=100} on the sync client, plus asyncio legs with paging -
normalized (scores compared bit-exact as float32, vectors by digest). All arms byte-identical to
the base.
and the same 6 environment-only timezone failures with no server binary).
flake8(the pinned3.8.4) reports no new finding on any changed file.
client's segment-parameter leg (
max_segments=1, feature bit 41) and the engine consolidated26 -> 1 as asked.
Behaviour notes for review
(key, value)row was alist(anasyncio.gatherartifact); it is nowa
tuple, matching the sync cursor.pygridgain.api.sql.vector()level, the legacydatavalue changes from a dict to a listof
(key, value)tuples (wire order kept). Cursor-level behaviour is unchanged.gg-50943(GG-50943 Support vector storage, segment parameters and INT8 on the thin client #83), which merges first; this PR then retargets to master.