Skip to content

Comments

Remove query-time usage of ByteSequence::slice in PQVectors to reduce object allocations#403

Merged
marianotepper merged 1 commit intodatastax:mainfrom
michaeljmarshall:reduce-bytesequence-slice-usage-on-query-path
Mar 21, 2025
Merged

Remove query-time usage of ByteSequence::slice in PQVectors to reduce object allocations#403
marianotepper merged 1 commit intodatastax:mainfrom
michaeljmarshall:reduce-bytesequence-slice-usage-on-query-path

Conversation

@michaeljmarshall
Copy link
Member

With #370, I introduced the ByteSequence::slice method and started using it on the query path. The PR replaces long lived pq vector ByteSequences with short lived slices of larger ByteSequences created at query time. However, profiling reveals that these objects get created very often for certain workloads, especially brute force workloads in Cassandra. After analyzing the code, it looks trivial to migrate to a solution where we do not create a slice object per access to a pq vector. The main downside is the addition of an offset and a length argument to many methods and the fact that these changes feel brittle, if we want to maintain the slice() construct, which I think we do for the simplicity of non-query path code. However, given the performance implications and the fact that the interface changes are all internal to jvector, I think this change is worth considering.

@michaeljmarshall
Copy link
Member Author

I keep seeing failures for:

Test2DThreshold.testThreshold20k:44->testThreshold:74 visited 75.38348% of the vectors, which is more than 75.0%

This doesn't seem related and feels potentially flaky since the tests pass on retry.

@marianotepper marianotepper requested review from jkni and removed request for jbellis and jkni March 20, 2025 21:14
@jkni
Copy link
Contributor

jkni commented Mar 21, 2025

Looks reasonable to me -- I agree that this is a bit brittle and suggests that we might want to revisit the API in the future, but I wouldn't call it disproportionately awkward/brittle compared to other performance-based sacrifices we've made around these code paths.

@marianotepper
Copy link
Contributor

I agree the comments above and the PR looks good to me. I suggest we open an issue about the interface so that we do not forget.

michaeljmarshall added a commit to datastax/cassandra that referenced this pull request Mar 21, 2025
@marianotepper marianotepper merged commit 0962ddb into datastax:main Mar 21, 2025
6 checks passed
@michaeljmarshall michaeljmarshall deleted the reduce-bytesequence-slice-usage-on-query-path branch March 21, 2025 21:06
jshook added a commit that referenced this pull request May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants