STAR-1706 Connect back using preferred address#561
Merged
Conversation
Port CNDB-4855 Fixed streaming to connect back using peer preferred address instead of Channel#remoteAddress
k-rus
force-pushed
the
star-1706-preferred
branch
from
9342660 to
cd1967f
Compare
jasonstack
approved these changes
Oct 14, 2022
| @@ -244,6 +244,25 @@ public StreamSession(StreamOperation streamOperation, InetAddressAndPort peer, S | |||
| logger.debug("Creating stream session to {} as {}", template, isFollower ? "follower" : "initiator"); | |||
There was a problem hiding this comment.
nit: how about refactoring the existing constructor as this(streamOperation, peer, peer, factory, isFollower, index, pendingRepair, previewKind);?
Member
Author
There was a problem hiding this comment.
I do like the idea to refactor it, but I don't like to repeat the same variable. I have an idea to provide template instead.
Member
Author
There was a problem hiding this comment.
@jasonstack Thanks for paying attention to duplicated code. I refactored now, see 2fdb14e
|
Kudos, SonarCloud Quality Gate passed! |
michaeljmarshall
added a commit
that referenced
this pull request
Feb 20, 2026
…2042) ### What is the issue Fixes: https://github.com/riptano/cndb/issues/15527 CNDB test PR: https://github.com/riptano/cndb/pull/16797 ### What does this PR fix and why was it fixed This PR upgrades jvector, which brings several improvements. Here are the git commits brought in: ``` 8b3e93cf (tag: 4.0.0-rc.8) chore: update changelog for 4.0.0-rc.8 (#627) 9d0488e5 release 4.0.0-rc.8 (#626) 570bd118 Refactor parallel writer (#608) 20c348ec Move buffer position in ByteBufferIndexWriter#writeFloats (#607) d9ddce51 Ensure extractTrainingVectors return a list of at most MAX_PQ_TRAINING_SET_SIZE (#610) d663b4f7 add config options for regression testing (#609) 7e493eee On-disk index cache for the Grid benchmark harness (#612) e263cc80 Improved dataset loading; fixes, safeties, diagnostics, and better feedback (#613) 6b235ce7 bump to next SNAPSHOT (#605) 84bf5708 (tag: 4.0.0-rc.7) chore: update changelog for 4.0.0-rc.7 (#604) fceeb885 release 4.0.0-rc.7 (#603) 51807cba add protection against bad ordinal mappings (#602) 6ca3b5e2 adding memory and disk usage stats to bench tests (#591) a66fd914 Fix OnDiskGraphIndex#ramBytesUsed NPE (#588) 0ca5a392 Move float bulk-write into IndexWriter to enforce endianness (#577) a6c6c09b Add diversityScoreFunctionFor to avoid creation of wrapper object (#592) 977c21d4 Relax the threshold of a flaky test related to an experimental feature (#598) fa808d69 adding average nodes visited to benchmark tests (#552) 3bd15e70 Virtualize and Modularize DataSetLoader logic (#593) 42259e9f Speed up ivec reads by buffering (#584) f967f1c9 virtualize DataSet (#589) 55f902f4 turn off parallel writes in grid (#582) 019a241d Parallelize graph writes (#542) 02fea879 Save allocation of a large array in PQVectors.encodeAndBuild (#574) 32a51821 javadoc for base [graph] (#548) 4eb607f8 javadoc for base [disk,exceptions] (#547) 30e8932c Enable the fused graph index (#561) d8848fc6 Start development on 4.0.0-rc.7-SNAPSHOT (#573) c57f3a62 (tag: 4.0.0-rc.6) chore: update changelog for 4.0.0-rc.6 (#572) 214b7c20 release 4.0.0-rc.6 (#571) e3686999 fix javadoc error (#570) 88669887 Ignoring testIncrementalInsertionFromOnDiskIndex_withNonIdentityOrdinalMapping and adding a TODO in buildAndMergeNewNodes (#569) 29a943e1 Computation of reconstruction errors for vector compressors (#567) d8e9cb16 Add NVQ paper in README (#560) d5cbe658 Add ImmutableGraphIndex.isHierarchical (#563) b484dae2 Harden tests for heap graph reconstruction (#543) 9471c57d Make the thresholds in TestLowCardinalityFiltering tighter (#559) 21e4a226 Begin development on 4.0.0-rc.6 (#558) 4f661d99 Revert "Start development on 4.0.0-rc.6-SNAPSHOT" fdee5779 Start development on 4.0.0-rc.6-SNAPSHOT ``` ### SAI Version Bump Adds a new sai on disk version: `fa` ### Fused PQ With this version, we are adding a new, experimental feature to write PQ vectors fused into the graph. In doing so, we are able to skip writing the PQ vectors to the PQ file, which results in significant memory savings since the PQ vectors in the `CassandraDiskAnn` graph searcher consumers `O(n)` memory based on the number of vectors and their quantized size. The fused pq vectors mostly fit within the page cache as we read the node and its neighbors from disk, so we see minimal latency reduction due to this change, though further testing is required to see the real impact. In order to enable fused pq, the runtime needs `cassandra.sai.latest.version=fa` or greater and `cassandra.sai.vector.enable_fused=true`. Note that because this feature is still experimental, `cassandra.sai.vector.enable_fused` defaults to `false`. Another experimental feature introduced in this commit via the jvector upgrade is parallel graph encoding and writing to disk. Writing the fused graph requires increased CPU time to encode the graph node and we write more bytes to disk, so this parallelism is likely necessary to keep vector index creation/compaction times down. The key configurations available with their associated defaults: ```java // When building a compaction graph, encode layer 0 nodes in parallel and subsequently use async io for writes. // This feature is experimental, so defaults to false. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_ENABLED("cassandra.sai.vector.encode_and_write_graph_in_parallel.enabled", "false"), // When parallel graph encoding is enabled, the number of threads to use for encoding. Defaults to 0, meaning // use all available processors as reported by the JVM. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_NUM_THREADS("cassandra.sai.vector.encode_and_write_graph_in_parallel.num_threads", "0"), // When parallel graph encoding is enabled, whether to use director buffers. Defaults to false, meaning heap // buffers are used. A buffer will be allocated per encoding thread. The size of each buffer is the size // of the encoded graph node at layer 0, which varies based on graph feature settings. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_USE_DIRECT_BUFFERS("cassandra.sai.vector.encode_and_write_graph_in_parallel.use_direct_buffers", "false"), ``` ### OnDiskVectorValues and OnDiskVectorValuesWriter `OnDiskVectorValues` is now in its own file and is now thread safe in order to account for some necessary implementation details within jvector. Added `OnDiskVectorValuesWriter` to improve test coverage and to abstract away the flush issues associated with `BufferedRandomAccessWriter` as described in datastax/jvector#562. ### Verification This PR also introduces new benchmarks as well as improved unit testing. The new benchmarks verify the performance of the `OnDiskVectorValues` and `OnDiskVectorValuesWriter` to confirm (at least directionally) the time associated with read and write operations. New tests have been added to verify that when we iterate over an sstable's rows, we are able to assert that the sstable's vector value's similarity to the one stored in the vector graph is ~1. This testing is valuable in that it confirms the row id to ordinal mapping is correct at every node. Previously, we relied on recall results to verify this for us. This new pattern allows us to confirm _every_ node, which is more thorough and removes most edge cases that might have led to partially correct graphs that may have achieved acceptable recall.
driftx
pushed a commit
that referenced
this pull request
Apr 27, 2026
…2042) Fixes: riptano/cndb#15527 CNDB test PR: riptano/cndb#16797 This PR upgrades jvector, which brings several improvements. Here are the git commits brought in: ``` 8b3e93cf (tag: 4.0.0-rc.8) chore: update changelog for 4.0.0-rc.8 (#627) 9d0488e5 release 4.0.0-rc.8 (#626) 570bd118 Refactor parallel writer (#608) 20c348ec Move buffer position in ByteBufferIndexWriter#writeFloats (#607) d9ddce51 Ensure extractTrainingVectors return a list of at most MAX_PQ_TRAINING_SET_SIZE (#610) d663b4f7 add config options for regression testing (#609) 7e493eee On-disk index cache for the Grid benchmark harness (#612) e263cc80 Improved dataset loading; fixes, safeties, diagnostics, and better feedback (#613) 6b235ce7 bump to next SNAPSHOT (#605) 84bf5708 (tag: 4.0.0-rc.7) chore: update changelog for 4.0.0-rc.7 (#604) fceeb885 release 4.0.0-rc.7 (#603) 51807cba add protection against bad ordinal mappings (#602) 6ca3b5e2 adding memory and disk usage stats to bench tests (#591) a66fd914 Fix OnDiskGraphIndex#ramBytesUsed NPE (#588) 0ca5a392 Move float bulk-write into IndexWriter to enforce endianness (#577) a6c6c09b Add diversityScoreFunctionFor to avoid creation of wrapper object (#592) 977c21d4 Relax the threshold of a flaky test related to an experimental feature (#598) fa808d69 adding average nodes visited to benchmark tests (#552) 3bd15e70 Virtualize and Modularize DataSetLoader logic (#593) 42259e9f Speed up ivec reads by buffering (#584) f967f1c9 virtualize DataSet (#589) 55f902f4 turn off parallel writes in grid (#582) 019a241d Parallelize graph writes (#542) 02fea879 Save allocation of a large array in PQVectors.encodeAndBuild (#574) 32a51821 javadoc for base [graph] (#548) 4eb607f8 javadoc for base [disk,exceptions] (#547) 30e8932c Enable the fused graph index (#561) d8848fc6 Start development on 4.0.0-rc.7-SNAPSHOT (#573) c57f3a62 (tag: 4.0.0-rc.6) chore: update changelog for 4.0.0-rc.6 (#572) 214b7c20 release 4.0.0-rc.6 (#571) e3686999 fix javadoc error (#570) 88669887 Ignoring testIncrementalInsertionFromOnDiskIndex_withNonIdentityOrdinalMapping and adding a TODO in buildAndMergeNewNodes (#569) 29a943e1 Computation of reconstruction errors for vector compressors (#567) d8e9cb16 Add NVQ paper in README (#560) d5cbe658 Add ImmutableGraphIndex.isHierarchical (#563) b484dae2 Harden tests for heap graph reconstruction (#543) 9471c57d Make the thresholds in TestLowCardinalityFiltering tighter (#559) 21e4a226 Begin development on 4.0.0-rc.6 (#558) 4f661d99 Revert "Start development on 4.0.0-rc.6-SNAPSHOT" fdee5779 Start development on 4.0.0-rc.6-SNAPSHOT ``` Adds a new sai on disk version: `fa` With this version, we are adding a new, experimental feature to write PQ vectors fused into the graph. In doing so, we are able to skip writing the PQ vectors to the PQ file, which results in significant memory savings since the PQ vectors in the `CassandraDiskAnn` graph searcher consumers `O(n)` memory based on the number of vectors and their quantized size. The fused pq vectors mostly fit within the page cache as we read the node and its neighbors from disk, so we see minimal latency reduction due to this change, though further testing is required to see the real impact. In order to enable fused pq, the runtime needs `cassandra.sai.latest.version=fa` or greater and `cassandra.sai.vector.enable_fused=true`. Note that because this feature is still experimental, `cassandra.sai.vector.enable_fused` defaults to `false`. Another experimental feature introduced in this commit via the jvector upgrade is parallel graph encoding and writing to disk. Writing the fused graph requires increased CPU time to encode the graph node and we write more bytes to disk, so this parallelism is likely necessary to keep vector index creation/compaction times down. The key configurations available with their associated defaults: ```java // When building a compaction graph, encode layer 0 nodes in parallel and subsequently use async io for writes. // This feature is experimental, so defaults to false. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_ENABLED("cassandra.sai.vector.encode_and_write_graph_in_parallel.enabled", "false"), // When parallel graph encoding is enabled, the number of threads to use for encoding. Defaults to 0, meaning // use all available processors as reported by the JVM. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_NUM_THREADS("cassandra.sai.vector.encode_and_write_graph_in_parallel.num_threads", "0"), // When parallel graph encoding is enabled, whether to use director buffers. Defaults to false, meaning heap // buffers are used. A buffer will be allocated per encoding thread. The size of each buffer is the size // of the encoded graph node at layer 0, which varies based on graph feature settings. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_USE_DIRECT_BUFFERS("cassandra.sai.vector.encode_and_write_graph_in_parallel.use_direct_buffers", "false"), ``` `OnDiskVectorValues` is now in its own file and is now thread safe in order to account for some necessary implementation details within jvector. Added `OnDiskVectorValuesWriter` to improve test coverage and to abstract away the flush issues associated with `BufferedRandomAccessWriter` as described in datastax/jvector#562. This PR also introduces new benchmarks as well as improved unit testing. The new benchmarks verify the performance of the `OnDiskVectorValues` and `OnDiskVectorValuesWriter` to confirm (at least directionally) the time associated with read and write operations. New tests have been added to verify that when we iterate over an sstable's rows, we are able to assert that the sstable's vector value's similarity to the one stored in the vector graph is ~1. This testing is valuable in that it confirms the row id to ordinal mapping is correct at every node. Previously, we relied on recall results to verify this for us. This new pattern allows us to confirm _every_ node, which is more thorough and removes most edge cases that might have led to partially correct graphs that may have achieved acceptable recall.
driftx
pushed a commit
that referenced
this pull request
Apr 28, 2026
…2042) Fixes: riptano/cndb#15527 CNDB test PR: riptano/cndb#16797 This PR upgrades jvector, which brings several improvements. Here are the git commits brought in: ``` 8b3e93cf (tag: 4.0.0-rc.8) chore: update changelog for 4.0.0-rc.8 (#627) 9d0488e5 release 4.0.0-rc.8 (#626) 570bd118 Refactor parallel writer (#608) 20c348ec Move buffer position in ByteBufferIndexWriter#writeFloats (#607) d9ddce51 Ensure extractTrainingVectors return a list of at most MAX_PQ_TRAINING_SET_SIZE (#610) d663b4f7 add config options for regression testing (#609) 7e493eee On-disk index cache for the Grid benchmark harness (#612) e263cc80 Improved dataset loading; fixes, safeties, diagnostics, and better feedback (#613) 6b235ce7 bump to next SNAPSHOT (#605) 84bf5708 (tag: 4.0.0-rc.7) chore: update changelog for 4.0.0-rc.7 (#604) fceeb885 release 4.0.0-rc.7 (#603) 51807cba add protection against bad ordinal mappings (#602) 6ca3b5e2 adding memory and disk usage stats to bench tests (#591) a66fd914 Fix OnDiskGraphIndex#ramBytesUsed NPE (#588) 0ca5a392 Move float bulk-write into IndexWriter to enforce endianness (#577) a6c6c09b Add diversityScoreFunctionFor to avoid creation of wrapper object (#592) 977c21d4 Relax the threshold of a flaky test related to an experimental feature (#598) fa808d69 adding average nodes visited to benchmark tests (#552) 3bd15e70 Virtualize and Modularize DataSetLoader logic (#593) 42259e9f Speed up ivec reads by buffering (#584) f967f1c9 virtualize DataSet (#589) 55f902f4 turn off parallel writes in grid (#582) 019a241d Parallelize graph writes (#542) 02fea879 Save allocation of a large array in PQVectors.encodeAndBuild (#574) 32a51821 javadoc for base [graph] (#548) 4eb607f8 javadoc for base [disk,exceptions] (#547) 30e8932c Enable the fused graph index (#561) d8848fc6 Start development on 4.0.0-rc.7-SNAPSHOT (#573) c57f3a62 (tag: 4.0.0-rc.6) chore: update changelog for 4.0.0-rc.6 (#572) 214b7c20 release 4.0.0-rc.6 (#571) e3686999 fix javadoc error (#570) 88669887 Ignoring testIncrementalInsertionFromOnDiskIndex_withNonIdentityOrdinalMapping and adding a TODO in buildAndMergeNewNodes (#569) 29a943e1 Computation of reconstruction errors for vector compressors (#567) d8e9cb16 Add NVQ paper in README (#560) d5cbe658 Add ImmutableGraphIndex.isHierarchical (#563) b484dae2 Harden tests for heap graph reconstruction (#543) 9471c57d Make the thresholds in TestLowCardinalityFiltering tighter (#559) 21e4a226 Begin development on 4.0.0-rc.6 (#558) 4f661d99 Revert "Start development on 4.0.0-rc.6-SNAPSHOT" fdee5779 Start development on 4.0.0-rc.6-SNAPSHOT ``` Adds a new sai on disk version: `fa` With this version, we are adding a new, experimental feature to write PQ vectors fused into the graph. In doing so, we are able to skip writing the PQ vectors to the PQ file, which results in significant memory savings since the PQ vectors in the `CassandraDiskAnn` graph searcher consumers `O(n)` memory based on the number of vectors and their quantized size. The fused pq vectors mostly fit within the page cache as we read the node and its neighbors from disk, so we see minimal latency reduction due to this change, though further testing is required to see the real impact. In order to enable fused pq, the runtime needs `cassandra.sai.latest.version=fa` or greater and `cassandra.sai.vector.enable_fused=true`. Note that because this feature is still experimental, `cassandra.sai.vector.enable_fused` defaults to `false`. Another experimental feature introduced in this commit via the jvector upgrade is parallel graph encoding and writing to disk. Writing the fused graph requires increased CPU time to encode the graph node and we write more bytes to disk, so this parallelism is likely necessary to keep vector index creation/compaction times down. The key configurations available with their associated defaults: ```java // When building a compaction graph, encode layer 0 nodes in parallel and subsequently use async io for writes. // This feature is experimental, so defaults to false. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_ENABLED("cassandra.sai.vector.encode_and_write_graph_in_parallel.enabled", "false"), // When parallel graph encoding is enabled, the number of threads to use for encoding. Defaults to 0, meaning // use all available processors as reported by the JVM. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_NUM_THREADS("cassandra.sai.vector.encode_and_write_graph_in_parallel.num_threads", "0"), // When parallel graph encoding is enabled, whether to use director buffers. Defaults to false, meaning heap // buffers are used. A buffer will be allocated per encoding thread. The size of each buffer is the size // of the encoded graph node at layer 0, which varies based on graph feature settings. SAI_ENCODE_AND_WRITE_VECTOR_GRAPH_IN_PARALLEL_USE_DIRECT_BUFFERS("cassandra.sai.vector.encode_and_write_graph_in_parallel.use_direct_buffers", "false"), ``` `OnDiskVectorValues` is now in its own file and is now thread safe in order to account for some necessary implementation details within jvector. Added `OnDiskVectorValuesWriter` to improve test coverage and to abstract away the flush issues associated with `BufferedRandomAccessWriter` as described in datastax/jvector#562. This PR also introduces new benchmarks as well as improved unit testing. The new benchmarks verify the performance of the `OnDiskVectorValues` and `OnDiskVectorValuesWriter` to confirm (at least directionally) the time associated with read and write operations. New tests have been added to verify that when we iterate over an sstable's rows, we are able to assert that the sstable's vector value's similarity to the one stored in the vector graph is ~1. This testing is valuable in that it confirms the row id to ordinal mapping is correct at every node. Previously, we relied on recall results to verify this for us. This new pattern allows us to confirm _every_ node, which is more thorough and removes most edge cases that might have led to partially correct graphs that may have achieved acceptable recall.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port CNDB-4855
Fixed streaming to connect back using peer preferred address instead of Channel#remoteAddress