Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function score multi values #5940

Closed
wants to merge 11 commits into from
90 changes: 58 additions & 32 deletions docs/reference/query-dsl/queries/function-score-query.asciidoc
Expand Up @@ -150,6 +150,46 @@ that is initialized with a `seed`.
}
--------------------------------------------------

===== Field Value factor
The `field_value_factor` function allows you to use a field from a document to
influence the score. It's similar to using the `script_score` function, however,
it avoids the overhead of scripting. If used on a multi-valued field, only the
first value of the field is used in calculations.

As an example, imagine you have a document indexed with a numeric `popularity`
field and wish in influence the score of a document with this field, an example
doing so would look like:

[source,js]
--------------------------------------------------
"field_value_factor": {
"field": "popularity",
"factor": 1.2,
"modifier": "sqrt"
}
--------------------------------------------------

Which will translate into the following forumla for scoring:

`sqrt(1.2 * doc['popularity'].value)`

There are a number of options for the `field_value_factor` function:

[cols="<,<",options="header",]
|=======================================================================
| Parameter |Description
|`field` |Field to be extracted from the document.
|`factor` |Optional factor to multiply the field value with, defaults to 1.
|`modifier` |Modifier to apply to the field value, can be one of: `none`, `log`,
`log1p`, `log2p`, `ln`, `ln1p`, `ln2p`, `square`, `sqrt`, or `reciprocal`.
Defaults to `none`.
|=======================================================================

Keep in mind that taking the log() of 0, or the square root of a negative number
is an illegal operation, and an exception will be thrown. Be sure to limit the
values of the field with a range filter to avoid this, or use `log1p` and
`ln1p`.

===== Decay functions

Decay functions score a document with a function that decays depending
Expand Down Expand Up @@ -246,45 +286,31 @@ In contrast to the normal and exponential decay, this function actually
sets the score to 0 if the field value exceeds twice the user given
scale value.

===== Field Value factor
The `field_value_factor` function allows you to use a field from a document to
influence the score. It's similar to using the `script_score` function, however,
it avoids the overhead of scripting. If used on a multi-valued field, only the
first value of the field is used in calculations.
===== Multiple values:

As an example, imagine you have a document indexed with a numeric `popularity`
field and wish in influence the score of a document with this field, an example
doing so would look like:
If a field used for computing the decay contains multiple values, per default the value closest to the origin is chosen for determining the distance.
This can be changed by setting `multi_value_mode`.

[horizontal]
`min`:: Distance is the minimum distance
`max`:: Distance is the maximum distance
`avg`:: Distance is the average distance
`sum`:: Distance is the sum of all distances

Example:

[source,js]
--------------------------------------------------
"field_value_factor": {
"field": "popularity",
"factor": 1.2,
"modifier": "sqrt"
}
"DECAY_FUNCTION": {
"FIELD_NAME": {
"origin": ...,
"scale": ...
},
"multi_value_mode": "avg"
}
--------------------------------------------------

Which will translate into the following forumla for scoring:

`sqrt(1.2 * doc['popularity'].value)`

There are a number of options for the `field_value_factor` function:

[cols="<,<",options="header",]
|=======================================================================
| Parameter |Description
|`field` |Field to be extracted from the document.
|`factor` |Optional factor to multiply the field value with, defaults to 1.
|`modifier` |Modifier to apply to the field value, can be one of: `none`, `log`,
`log1p`, `log2p`, `ln`, `ln1p`, `ln2p`, `square`, `sqrt`, or `reciprocal`.
Defaults to `none`.
|=======================================================================

Keep in mind that taking the log() of 0, or the square root of a negative number
is an illegal operation, and an exception will be thrown. Be sure to limit the
values of the field with a range filter to avoid this, or use `log1p` and
`ln1p`.

==== Detailed example

Expand Down
Expand Up @@ -29,7 +29,7 @@
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.index.Index;
import org.elasticsearch.index.IndexComponent;
import org.elasticsearch.index.fielddata.fieldcomparator.SortMode;
import org.elasticsearch.search.MultiValueMode;
import org.elasticsearch.index.fielddata.ordinals.GlobalOrdinalsBuilder;
import org.elasticsearch.index.mapper.FieldMapper;
import org.elasticsearch.index.mapper.MapperService;
Expand Down Expand Up @@ -97,7 +97,7 @@ public static MemoryStorageFormat getMemoryStorageHint(FieldDataType fieldDataTy
/**
* Comparator used for sorting.
*/
XFieldComparatorSource comparatorSource(@Nullable Object missingValue, SortMode sortMode);
XFieldComparatorSource comparatorSource(@Nullable Object missingValue, MultiValueMode sortMode);

/**
* Clears any resources associated with this field data.
Expand Down
Expand Up @@ -24,6 +24,7 @@
import org.apache.lucene.util.BytesRef;
import org.apache.lucene.util.UnicodeUtil;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -41,10 +42,10 @@ public class BytesRefFieldComparatorSource extends IndexFieldData.XFieldComparat
}

private final IndexFieldData<?> indexFieldData;
private final SortMode sortMode;
private final MultiValueMode sortMode;
private final Object missingValue;

public BytesRefFieldComparatorSource(IndexFieldData<?> indexFieldData, Object missingValue, SortMode sortMode) {
public BytesRefFieldComparatorSource(IndexFieldData<?> indexFieldData, Object missingValue, MultiValueMode sortMode) {
this.indexFieldData = indexFieldData;
this.sortMode = sortMode;
this.missingValue = missingValue;
Expand Down
Expand Up @@ -25,6 +25,7 @@
import org.elasticsearch.index.fielddata.BytesValues;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.ordinals.Ordinals;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand Down Expand Up @@ -54,7 +55,7 @@ public final class BytesRefOrdValComparator extends NestedWrappableComparator<By
@lucene.internal */
final long[] ords;

final SortMode sortMode;
final MultiValueMode sortMode;

/* Values for each slot.
@lucene.internal */
Expand Down Expand Up @@ -88,7 +89,7 @@ public final class BytesRefOrdValComparator extends NestedWrappableComparator<By
BytesRef top;
long topOrd;

public BytesRefOrdValComparator(IndexFieldData.WithOrdinals<?> indexFieldData, int numHits, SortMode sortMode, BytesRef missingValue) {
public BytesRefOrdValComparator(IndexFieldData.WithOrdinals<?> indexFieldData, int numHits, MultiValueMode sortMode, BytesRef missingValue) {
this.indexFieldData = indexFieldData;
this.sortMode = sortMode;
this.missingValue = missingValue;
Expand Down Expand Up @@ -395,11 +396,11 @@ else if (cmp > 0)
return -(low + 1);
}

static long getRelevantOrd(Ordinals.Docs readerOrds, int docId, SortMode sortMode) {
static long getRelevantOrd(Ordinals.Docs readerOrds, int docId, MultiValueMode sortMode) {
int length = readerOrds.setDocument(docId);
long relevantVal = sortMode.startLong();
long result = 0;
assert sortMode == SortMode.MAX || sortMode == SortMode.MIN;
assert sortMode == MultiValueMode.MAX || sortMode == MultiValueMode.MIN;
for (int i = 0; i < length; i++) {
result = relevantVal = sortMode.apply(readerOrds.nextOrd(), relevantVal);
}
Expand Down
Expand Up @@ -24,6 +24,7 @@
import org.apache.lucene.util.BytesRef;
import org.elasticsearch.index.fielddata.BytesValues;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -36,15 +37,15 @@
public final class BytesRefValComparator extends NestedWrappableComparator<BytesRef> {

private final IndexFieldData<?> indexFieldData;
private final SortMode sortMode;
private final MultiValueMode sortMode;
private final BytesRef missingValue;

private final BytesRef[] values;
private BytesRef bottom;
private BytesRef top;
private BytesValues docTerms;

BytesRefValComparator(IndexFieldData<?> indexFieldData, int numHits, SortMode sortMode, BytesRef missingValue) {
BytesRefValComparator(IndexFieldData<?> indexFieldData, int numHits, MultiValueMode sortMode, BytesRef missingValue) {
this.sortMode = sortMode;
values = new BytesRef[numHits];
this.indexFieldData = indexFieldData;
Expand Down
Expand Up @@ -20,6 +20,7 @@
package org.elasticsearch.index.fielddata.fieldcomparator;

import org.elasticsearch.index.fielddata.IndexNumericFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -29,7 +30,7 @@ public final class DoubleValuesComparator extends DoubleValuesComparatorBase<Dou

private final double[] values;

public DoubleValuesComparator(IndexNumericFieldData<?> indexFieldData, double missingValue, int numHits, SortMode sortMode) {
public DoubleValuesComparator(IndexNumericFieldData<?> indexFieldData, double missingValue, int numHits, MultiValueMode sortMode) {
super(indexFieldData, missingValue, sortMode);
assert indexFieldData.getNumericType().requiredBits() <= 64;
this.values = new double[numHits];
Expand Down
Expand Up @@ -22,6 +22,7 @@
import org.apache.lucene.search.FieldComparator;
import org.elasticsearch.index.fielddata.DoubleValues;
import org.elasticsearch.index.fielddata.IndexNumericFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -31,9 +32,9 @@ abstract class DoubleValuesComparatorBase<T extends Number> extends NumberCompar
protected final double missingValue;
protected double bottom;
protected DoubleValues readerValues;
protected final SortMode sortMode;
protected final MultiValueMode sortMode;

public DoubleValuesComparatorBase(IndexNumericFieldData<?> indexFieldData, double missingValue, SortMode sortMode) {
public DoubleValuesComparatorBase(IndexNumericFieldData<?> indexFieldData, double missingValue, MultiValueMode sortMode) {
this.indexFieldData = indexFieldData;
this.missingValue = missingValue;
this.sortMode = sortMode;
Expand Down
Expand Up @@ -24,6 +24,7 @@
import org.elasticsearch.common.Nullable;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.IndexNumericFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -33,9 +34,9 @@ public class DoubleValuesComparatorSource extends IndexFieldData.XFieldComparato

private final IndexNumericFieldData<?> indexFieldData;
private final Object missingValue;
private final SortMode sortMode;
private final MultiValueMode sortMode;

public DoubleValuesComparatorSource(IndexNumericFieldData<?> indexFieldData, @Nullable Object missingValue, SortMode sortMode) {
public DoubleValuesComparatorSource(IndexNumericFieldData<?> indexFieldData, @Nullable Object missingValue, MultiValueMode sortMode) {
this.indexFieldData = indexFieldData;
this.missingValue = missingValue;
this.sortMode = sortMode;
Expand Down
Expand Up @@ -19,6 +19,7 @@
package org.elasticsearch.index.fielddata.fieldcomparator;

import org.elasticsearch.index.fielddata.IndexNumericFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -28,7 +29,7 @@ public final class FloatValuesComparator extends DoubleValuesComparatorBase<Floa

private final float[] values;

public FloatValuesComparator(IndexNumericFieldData<?> indexFieldData, float missingValue, int numHits, SortMode sortMode) {
public FloatValuesComparator(IndexNumericFieldData<?> indexFieldData, float missingValue, int numHits, MultiValueMode sortMode) {
super(indexFieldData, missingValue, sortMode);
assert indexFieldData.getNumericType().requiredBits() <= 32;
this.values = new float[numHits];
Expand Down
Expand Up @@ -23,6 +23,7 @@
import org.elasticsearch.common.Nullable;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.IndexNumericFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -32,9 +33,9 @@ public class FloatValuesComparatorSource extends IndexFieldData.XFieldComparator

private final IndexNumericFieldData<?> indexFieldData;
private final Object missingValue;
private final SortMode sortMode;
private final MultiValueMode sortMode;

public FloatValuesComparatorSource(IndexNumericFieldData<?> indexFieldData, @Nullable Object missingValue, SortMode sortMode) {
public FloatValuesComparatorSource(IndexNumericFieldData<?> indexFieldData, @Nullable Object missingValue, MultiValueMode sortMode) {
this.indexFieldData = indexFieldData;
this.missingValue = missingValue;
this.sortMode = sortMode;
Expand Down
Expand Up @@ -26,6 +26,7 @@
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.index.fielddata.GeoPointValues;
import org.elasticsearch.index.fielddata.IndexGeoPointFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -40,15 +41,15 @@ public class GeoDistanceComparator extends NumberComparatorBase<Double> {
protected final DistanceUnit unit;
protected final GeoDistance geoDistance;
protected final GeoDistance.FixedSourceDistance fixedSourceDistance;
protected final SortMode sortMode;
protected final MultiValueMode sortMode;
private static final Double MISSING_VALUE = Double.MAX_VALUE;

private final double[] values;
private double bottom;

private GeoDistanceValues geoDistanceValues;

public GeoDistanceComparator(int numHits, IndexGeoPointFieldData<?> indexFieldData, double lat, double lon, DistanceUnit unit, GeoDistance geoDistance, SortMode sortMode) {
public GeoDistanceComparator(int numHits, IndexGeoPointFieldData<?> indexFieldData, double lat, double lon, DistanceUnit unit, GeoDistance geoDistance, MultiValueMode sortMode) {
this.values = new double[numHits];
this.indexFieldData = indexFieldData;
this.lat = lat;
Expand Down Expand Up @@ -166,9 +167,9 @@ public double computeDistance(int doc) {
// Deals with more than one geo point per document
private static final class MV extends GeoDistanceValues {

private final SortMode sortMode;
private final MultiValueMode sortMode;

MV(GeoPointValues readerValues, GeoDistance.FixedSourceDistance fixedSourceDistance, SortMode sortMode) {
MV(GeoPointValues readerValues, GeoDistance.FixedSourceDistance fixedSourceDistance, MultiValueMode sortMode) {
super(readerValues, fixedSourceDistance);
this.sortMode = sortMode;
}
Expand Down
Expand Up @@ -25,6 +25,7 @@
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.index.fielddata.IndexFieldData;
import org.elasticsearch.index.fielddata.IndexGeoPointFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -37,9 +38,9 @@ public class GeoDistanceComparatorSource extends IndexFieldData.XFieldComparator
private final double lon;
private final DistanceUnit unit;
private final GeoDistance geoDistance;
private final SortMode sortMode;
private final MultiValueMode sortMode;

public GeoDistanceComparatorSource(IndexGeoPointFieldData<?> indexFieldData, double lat, double lon, DistanceUnit unit, GeoDistance geoDistance, SortMode sortMode) {
public GeoDistanceComparatorSource(IndexGeoPointFieldData<?> indexFieldData, double lat, double lon, DistanceUnit unit, GeoDistance geoDistance, MultiValueMode sortMode) {
this.indexFieldData = indexFieldData;
this.lat = lat;
this.lon = lon;
Expand Down
Expand Up @@ -19,6 +19,7 @@
package org.elasticsearch.index.fielddata.fieldcomparator;

import org.elasticsearch.index.fielddata.IndexNumericFieldData;
import org.elasticsearch.search.MultiValueMode;

import java.io.IOException;

Expand All @@ -28,7 +29,7 @@ public final class LongValuesComparator extends LongValuesComparatorBase<Long> {

private final long[] values;

public LongValuesComparator(IndexNumericFieldData<?> indexFieldData, long missingValue, int numHits, SortMode sortMode) {
public LongValuesComparator(IndexNumericFieldData<?> indexFieldData, long missingValue, int numHits, MultiValueMode sortMode) {
super(indexFieldData, missingValue, sortMode);
this.values = new long[numHits];
assert indexFieldData.getNumericType().requiredBits() <= 64;
Expand Down