Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Point based optimization for custom comparators #8168

Merged
merged 11 commits into from
Jun 29, 2023
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Add descending order search optimization through reverse segment read. ([#7967](https://github.com/opensearch-project/OpenSearch/pull/7967))
- Update components of segrep backpressure to support remote store. ([#8020](https://github.com/opensearch-project/OpenSearch/pull/8020))
- Make remote cluster connection setup in async ([#8038](https://github.com/opensearch-project/OpenSearch/pull/8038))
- Enable Point based optimization for custom comparators ([#8168](https://github.com/opensearch-project/OpenSearch/pull/8168))

### Dependencies
- Bump `com.azure:azure-storage-common` from 12.21.0 to 12.21.1 (#7566, #7814)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,14 @@ public final BucketedSort newBucketedSort(
return newBucketedSort(getNumericType(), bigArrays, missingValue, sortMode, nested, sortOrder, format, bucketSize, extra);
}

/**
* Should enable BKD based skipping values sort optimization if target numeric type is matching
* @param targetNumericType
*/
public boolean enableSortOptimizationForCustomComparators(NumericType targetNumericType) {
return this.getNumericType() == targetNumericType;
}

/**
* Build a {@link XFieldComparatorSource} matching the parameters.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,13 @@ public FieldComparator<?> newComparator(String fieldname, int numHits, boolean e
final double dMissingValue = (Double) missingObject(missingValue, reversed);
// NOTE: it's important to pass null as a missing value in the constructor so that
// the comparator doesn't check docsWithField since we replace missing values in select()
return new DoubleComparator(numHits, null, null, reversed, false) {
return new DoubleComparator(
numHits,
fieldname,
null,
reversed,
indexFieldData.enableSortOptimizationForCustomComparators(IndexNumericFieldData.NumericType.DOUBLE)
) {
@Override
public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws IOException {
return new DoubleLeafComparator(context) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,13 @@ public FieldComparator<?> newComparator(String fieldname, int numHits, boolean e
final float fMissingValue = (Float) missingObject(missingValue, reversed);
// NOTE: it's important to pass null as a missing value in the constructor so that
// the comparator doesn't check docsWithField since we replace missing values in select()
return new FloatComparator(numHits, null, null, reversed, false) {
return new FloatComparator(
numHits,
fieldname,
null,
reversed,
indexFieldData.enableSortOptimizationForCustomComparators(IndexNumericFieldData.NumericType.FLOAT)
) {
@Override
public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws IOException {
return new FloatLeafComparator(context) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,13 @@ public FieldComparator<?> newComparator(String fieldname, int numHits, boolean e
final int iMissingValue = (Integer) missingObject(missingValue, reversed);
// NOTE: it's important to pass null as a missing value in the constructor so that
// the comparator doesn't check docsWithField since we replace missing values in select()
return new IntComparator(numHits, null, null, reversed, false) {
return new IntComparator(
numHits,
fieldname,
null,
reversed,
indexFieldData.enableSortOptimizationForCustomComparators(IndexNumericFieldData.NumericType.INT)
) {
@Override
public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws IOException {
return new IntLeafComparator(context) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,13 @@ public FieldComparator<?> newComparator(String fieldname, int numHits, boolean e
final long lMissingValue = (Long) missingObject(missingValue, reversed);
// NOTE: it's important to pass null as a missing value in the constructor so that
// the comparator doesn't check docsWithField since we replace missing values in select()
return new LongComparator(numHits, null, null, reversed, false) {
return new LongComparator(
numHits,
fieldname,
null,
reversed,
indexFieldData.enableSortOptimizationForCustomComparators(IndexNumericFieldData.NumericType.LONG)
reta marked this conversation as resolved.
Show resolved Hide resolved
) {
@Override
public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws IOException {
return new LongLeafComparator(context) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,13 @@ public FieldComparator<?> newComparator(String fieldname, int numHits, boolean e
assert indexFieldData == null || fieldname.equals(indexFieldData.getFieldName());

final BigInteger ulMissingValue = (BigInteger) missingObject(missingValue, reversed);
return new UnsignedLongComparator(numHits, null, null, reversed, false) {
return new UnsignedLongComparator(
numHits,
fieldname,
null,
reversed,
indexFieldData.enableSortOptimizationForCustomComparators(IndexNumericFieldData.NumericType.UNSIGNED_LONG)
) {
@Override
public LeafFieldComparator getLeafComparator(LeafReaderContext context) throws IOException {
return new UnsignedLongLeafComparator(context) {
Expand Down