ATLAS-5194: Add pagination while fetching relationships of an entity #510
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Introduces
searchRelatedEntitiesV2with graph-layer pagination support, achieving a significant performance improvement for large datasets and deep pagination scenarios (offset > 1000). The new method selects between database-level pagination (for unsorted queries) and traditional Gremlin traversal (for sorted queries), while maintaining full backward compatibility with the existingsearchRelatedEntitiesmethodWhy It Was Needed?
The earlier
searchRelatedEntitiesimplementation applies default sorting by the name attribute in ascending order when no sort criteria are provided. This requires loading all related entities into memory before sorting and pagination can be applied. As a result, queries involving a large number of relationships or high offset values suffer from significant performance degradation, leading to long response times and, in some cases, timeouts.How was this patch tested?
Performance validation was conducted on test databases with 100, 1K, 10K, and 50K entities(relationships), measuring server-side execution time from audit logs across multiple manual test scenarios.
Performance Impact (Accurate Data)
• No regression for sorted queries