Skip to content

GG-49932 Vector query fast paths: one-pass response decode, shared ctypes classes, sane paging default - #84

Closed
PakhomovAlexander wants to merge 3 commits into
gg-50943from
gg-49932
Closed

GG-49932 Vector query fast paths: one-pass response decode, shared ctypes classes, sane paging default#84
PakhomovAlexander wants to merge 3 commits into
gg-50943from
gg-49932

Conversation

@PakhomovAlexander

@PakhomovAlexander PakhomovAlexander commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Ticket: GG-49932

Vector-query response deserialization was the bulk of this client's per-query cost. Profiling on a
settled 100k x 1536-d index (server = CE/EE master + gg-50942 + gg-50943, the branches that merge
next) put ~24% of client CPU in dynamic ctypes class creation - one type() call per element per
row - and the rest in the triple walk: page parse, to_python, then a second full parse of every
wrapped binary value in unwrap_binary.

Three commits, layered so each is measurable on its own:

1. Cache ctypes classes per shape; page_size defaults to k; reject non-positive k.
cached_c_type generalizes the GG-50683 memo to the six sites that still built a class per parse:
wrapped payloads, binary objects and their schema elements, primitive arrays, map/collection
containers, struct arrays, response envelopes. The classes are only ever read, never mutated, so
one shared class per shape is safe; shapes recur because ctypes caches array types and the leaf
classes come from the same cache. The page_size=1 default meant one server round trip per result
row (20 recv calls per k=10 query); a vector query returns at most k rows, so the default is now
one page holding the whole result. k < 1 raises ValueError before any I/O, mirroring the
server-side bound from GG-49668 (the 10000 ceiling stays server-side - it is configurable there).

2. Decode vector query responses in one pass (VectorResponse).
The same pattern as the existing SQLResponse: rows leave parse() as final Python values, read
straight off the response buffer - no per-element ctypes classes, no payload blob copy, no second
parse. Direct readers cover what vector rows actually carry (long/int/double/string keys, binary
objects with primitive or float-array fields, raw float scores); any other element falls back to
the generic machinery at the same stream position, one element at a time. Covers the flagged and
the legacy (v1 map) row layouts, sync and asyncio. The vector cursors stop calling unwrap_binary.

3. Server-free tests: the k guard, the page_size default, and the shared-class cache.

Measured (GCP VM, one bracketed invocation: base -> 1 -> 1+2 -> base again)

2000 queries per leg after warm-up, ef=40, dbpedia-openai 100k, single-segment index, drift
bracket 0.6-5%. Mean ms per query (QPS) / client CPU ms:

leg base (#83) commit 1 commits 1+2
(key, value) k=10 5.54 (181) / 2.33 4.40 (227) / 1.33 4.05 (247) / 1.01
(key, value, score) 5.28 (189) / 2.24 4.47 (224) / 1.40 4.03 (248) / 1.02
keys only 3.22 (311) / 0.48 3.10 (323) / 0.39 3.01 (332) / 0.30
(key, value) k=100 23.50 (43) / 16.6 16.93 (59) / 10.0 13.92 (72) / 7.03
old default (page_size=1), k=10 8.70 (115) 6.92 (145) 6.38 (157)

The baseline also retained 562-5422 objects per query (the per-row classes awaiting gc); both
optimized arms retain none.

Correctness

  • Row-equality gate before any timing: 640 row-lists over 16 legs per arm - four row shapes x
    {k=10, k=10 with page_size=3 (paging), k=100} on the sync client, plus asyncio legs with paging -
    normalized (scores compared bit-exact as float32, vectors by digest). All arms byte-identical to
    the base.
  • Offline suite: identical outcome to the base tree on the same machine (159 passed, 38 skipped,
    and the same 6 environment-only timezone failures with no server binary). flake8 (the pinned
    3.8.4) reports no new finding on any changed file.
  • e2e against a live UE node: the A/B above, plus the index for it was created through this
    client's segment-parameter leg (max_segments=1, feature bit 41) and the engine consolidated
    26 -> 1 as asked.

Behaviour notes for review

  • The asyncio legacy (key, value) row was a list (an asyncio.gather artifact); it is now
    a tuple, matching the sync cursor.
  • At the pygridgain.api.sql.vector() level, the legacy data value changes from a dict to a list
    of (key, value) tuples (wire order kept). Cursor-level behaviour is unchanged.
  • Base is gg-50943 (GG-50943 Support vector storage, segment parameters and INT8 on the thin client #83), which merges first; this PR then retargets to master.

@PakhomovAlexander

Copy link
Copy Markdown
Contributor Author

Pre-review gate: the private EE suite (ggprivate-python-thin-client, branch gg-50205 — the GG-49286 e2e matrix plus the older vector tests) passes 53/53 with this branch installed as the client, against a live UE node built from CE/EE master + gg-50942 + gg-50943 (build.sh --assembly ent). That covers feature negotiation, all four row shapes, paging-keeps-shape on the sync and asyncio clients, and the v1 legacy path — the exact surfaces this PR rewrites.

@PakhomovAlexander

Copy link
Copy Markdown
Contributor Author

Superseded by the split requested for review clarity: PR #85 (shared ctypes-class cache + paging default + k guard) and PR #86 (one-pass vector response decode), stacked. Same ticket, same code, one review unit each. Numbers and verification carry over; each PR's description is self-contained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants