Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new 'exact' DSL query #92351

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
5 changes: 5 additions & 0 deletions docs/changelog/92351.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
pr: 92351
summary: Add new 'exact' DSL query
area: Search
type: feature
issues: []
46 changes: 46 additions & 0 deletions docs/reference/query-dsl/exact-query.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
[[query-dsl-exact-query]]
=== Exact query
++++
<titleabbrev>Exact</titleabbrev>
++++

Returns documents that contain an exact value for a field.

For most field types, this behaves the same as a <<query-dsl-term-query, term query>>. For
analyzed fields such as <<text-field-type, text>> or <<keyword-field-type, keyword>>
fields with a normalizer configured, it will only return documents where
the query value exactly matches the entire content of the field in the source.

[[exact-query-ex-request]]
==== Example request

[source,console]
----
GET /_search
{
"query": {
"exact": {
"text": "this is some text"
}
}
}
----

This will return documents that have precisely `this is some text` in their `text` field,
but it will not return documents with values of `some text`, `this is some text again`, or
`This is some Text`.

[[exact-query-top-level-params]]
==== Top-level parameters for `exact`
`field`::
(Required, string) Name of the field you wish to search.
+

[[exact-query-notes]]
==== Notes

[[exact-query-notes-multivalued]]
===== Multi-valued fields

If a document has multiple values for a field, then `exact` will return if it matches
any one of those values.
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ protected boolean supportsStoredFields() {
return false;
}

@Override
protected boolean supportsExactQuery() {
return false;
}

@Override
protected void registerParameters(ParameterChecker checker) throws IOException {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
import org.elasticsearch.index.mapper.SourceValueFetcher;
import org.elasticsearch.index.mapper.StringFieldType;
import org.elasticsearch.index.mapper.StringStoredFieldFieldLoader;
import org.elasticsearch.index.mapper.TextFieldExactQuery;
import org.elasticsearch.index.mapper.TextFieldMapper;
import org.elasticsearch.index.mapper.TextFieldMapper.TextFieldType;
import org.elasticsearch.index.mapper.TextParams;
Expand Down Expand Up @@ -246,6 +247,11 @@ public Query termQuery(Object value, SearchExecutionContext context) {
return new ConstantScoreQuery(super.termQuery(value, context));
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
return new TextFieldExactQuery(this, context.getForField(this, FielddataOperation.SOURCE), value.toString());
}

@Override
public Query fuzzyQuery(
Object value,
Expand Down Expand Up @@ -319,7 +325,7 @@ public Query phrasePrefixQuery(TokenStream stream, int slop, int maxExpansions,

@Override
public IndexFieldData.Builder fielddataBuilder(FieldDataContext fieldDataContext) {
if (fieldDataContext.fielddataOperation() != FielddataOperation.SCRIPT) {
if (fieldDataContext.fielddataOperation() == FielddataOperation.SEARCH) {
throw new IllegalArgumentException(CONTENT_TYPE + " fields do not support sorting and aggregations");
}
if (textFieldType.isSyntheticSource()) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,11 @@ public Query existsQuery(SearchExecutionContext context) {
return new TermQuery(new Term("_feature", name()));
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Field [" + name() + "] of type [" + typeName() + "] doesn't support exact queries");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: maybe use UnsupportedOperationException here and all other similar implementations?

}

@Override
public IndexFieldData.Builder fielddataBuilder(FieldDataContext fieldDataContext) {
throw new IllegalArgumentException("[rank_feature] fields do not support sorting, scripting or aggregating");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,11 @@ public boolean positiveScoreImpact() {
return positiveScoreImpact;
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Field [" + name() + "] of type [" + typeName() + "] doesn't support exact queries");
}

@Override
public Query existsQuery(SearchExecutionContext context) {
throw new IllegalArgumentException("[rank_features] fields do not support [exists] queries");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,11 @@ private void checkForPositions() {
}
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Field [" + name() + "] of type [" + typeName() + "] doesn't support exact queries");
}

@Override
public Query phraseQuery(TokenStream stream, int slop, boolean enablePositionIncrements, SearchExecutionContext context)
throws IOException {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ protected Collection<Plugin> getPlugins() {

@Override
protected Object getSampleValueForDocument() {
return "value";
return "text value";
}

public void testExistsStandardSource() throws IOException {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ protected boolean supportsIgnoreMalformed() {
return false;
}

@Override
protected boolean supportsExactQuery() {
return false;
}

@Override
protected Collection<? extends Plugin> getPlugins() {
return List.of(new MapperExtrasPlugin());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,11 @@ protected boolean supportsIgnoreMalformed() {
return false;
}

@Override
protected boolean supportsExactQuery() {
return false;
}

@Override
protected void registerParameters(ParameterChecker checker) throws IOException {
checker.registerConflictCheck("positive_score_impact", b -> b.field("positive_score_impact", false));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,11 @@ protected Object getSampleValueForDocument() {
return "new york city";
}

@Override
protected boolean supportsExactQuery() {
return false; // TODO: support this? Needs fielddata script access
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before adding this and other TODOs that are difficult to remove without more context in the future, should we collect reasons for potential support of this and other field types in a follow up issue? To me it gives a better overview, has more space for context than a short TODO and is more visible in the backlog.
For this particular case I'm struggling to see the need for support.

}

@Override
protected Collection<? extends Plugin> getPlugins() {
return List.of(new MapperExtrasPlugin());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,11 @@ public ValueFetcher valueFetcher(SearchExecutionContext context, String format)
public Query termQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Murmur3 fields are not searchable: [" + name() + "]");
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Field [" + name() + "] of type [" + typeName() + "] doesn't support exact queries");
}
}

protected Murmur3FieldMapper(String simpleName, MappedFieldType mappedFieldType, MultiFields multiFields, CopyTo copyTo) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ protected boolean supportsIgnoreMalformed() {
return false;
}

@Override
protected boolean supportsExactQuery() {
return false;
}

@Override
protected SyntheticSourceSupport syntheticSourceSupport(boolean ignoreMalformed) {
throw new AssumptionViolatedException("not supported");
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
setup:
- skip:
version: " - 8.7.0"
reason: exact query introduced in 8.7

- do:
indices.create:
index: exact_query_test
body:
settings:
analysis:
normalizer:
lowercase:
type: custom
filter: lowercase
mappings:
properties:
text:
type: text
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if it would also make sense to add a test with a text fields with a non-standard analyzer, e.g. something that changes case here

keyword:
type: keyword
nkeyword:
type: keyword
normalizer: lowercase

- do:
bulk:
refresh: true
body:
- '{ "index" : { "_index" : "exact_query_test", "_id" : "1" } }'
- '{ "text" : "here is some text", "keyword" : ["foo", "bar"], "nkeyword" : "hello" }'
- '{ "index" : { "_index" : "exact_query_test", "_id" : "2" } }'
- '{ "text" : "here is some different text", "keyword" : ["foo", "baz"], "nkeyword" : "HELLO" }'
- '{ "index" : { "_index" : "exact_query_test", "_id" : "3" } }'
- '{ "text" : "there is some text", "keyword" : "bar", "nkeyword" : "HELLO" }'
- '{ "index" : { "_index" : "exact_query_test", "_id" : "4" } }'
- '{ "text" : "here is some text", "keyword" : ["foo", "bar"], "nkeyword" : "hello" }'
- '{ "index" : { "_index" : "exact_query_test", "_id" : "5" } }'
- '{ "text" : "text", "keyword" : "foo" }'
- '{ "index" : { "_index" : "exact_query_test", "_id" : "6" } }'
- '{ "text" : "here", "keyword" : ["foo", "bar"], "nkeyword" : "hello" }'
- '{ "index" : { "_index" : "exact_query_test", "_id" : "7" } }'
- '{ "text" : "there is some text", "keyword" : ["foo", "bar"] }'
- '{ "index" : { "_index" : "exact_query_test", "_id" : "8" } }'
- '{ "text" : "here is some text", "keyword" : ["baz", "bar"], "nkeyword" : "Hello" }'

---
"test exact query on text fields":
- do:
search:
index: exact_query_test
body:
query:
exact:
text: "here is some text"

- match: { hits.total: 3 }

- do:
search:
index: exact_query_test
body:
query:
exact:
text: "here"

- match: { hits.total: 1 }

---
"test exact query on keyword fields":
- do:
search:
index: exact_query_test
body:
query:
exact:
keyword: baz

- match: { hits.total: 2 }

---
"test exact query on normalized keyword fields":
- do:
search:
index: exact_query_test
body:
query:
exact:
nkeyword: hello

- match: { hits.total: 3 }

- do:
search:
index: exact_query_test
body:
query:
exact:
nkeyword: HELLO

- match: { hits.total: 2 }
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,11 @@ public final Query termQuery(Object value, SearchExecutionContext context) {
);
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Field [" + name() + "] of type [" + typeName() + "] doesn't support exact queries");
}

/**
* Gets the formatter by name.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,11 @@ public IndexFieldData.Builder fielddataBuilder(FieldDataContext fieldDataContext
public Query termQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Binary fields do not support searching");
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Field [" + name() + "] of type [" + typeName() + "] doesn't support exact queries");
}
}

private final boolean stored;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,14 @@ public Query termQuery(Object value, SearchExecutionContext context) {
}
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
if (normalizer == Lucene.KEYWORD_ANALYZER) {
return super.exactQuery(value, context);
}
return new TextFieldExactQuery(this, context.getForField(this, FielddataOperation.SOURCE), value.toString());
}

@Override
public Query termsQuery(Collection<?> values, SearchExecutionContext context) {
failIfNotIndexedNorDocValuesFallback(context);
Expand Down Expand Up @@ -703,11 +711,8 @@ public IndexFieldData.Builder fielddataBuilder(FieldDataContext fieldDataContext
failIfNoDocValues();
return fieldDataFromDocValues();
}
if (operation != FielddataOperation.SCRIPT) {
throw new IllegalStateException("unknown operation [" + operation.name() + "]");
}

if (hasDocValues()) {
if (operation != FielddataOperation.SOURCE && hasDocValues()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I read this correctly, for operation == FielddataOperation.SEARCH this falls through until here now, formerly it would have raised an exception. Is this intended?

return fieldDataFromDocValues();
}
if (isSyntheticSource) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -85,8 +85,12 @@ public MappedFieldType(
* field data from and generate a representation of doc values.
*/
public enum FielddataOperation {
// Fielddata to be used as part of a search or aggregation
SEARCH,
SCRIPT
// Fielddata to be used as part of a script
SCRIPT,
// Fielddata that must be read from source
SOURCE
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For my understanding: I see this constant set in FieldDataContext three times but I cannot find a spot where the value is actually used to execute - say another code path - than the existing ones. Is it solely here to mark field data usage other than SCRIPT or SEARCH?

}

/**
Expand Down Expand Up @@ -214,6 +218,13 @@ public Query termQueryCaseInsensitive(Object value, @Nullable SearchExecutionCon
);
}

/**
* Generates a query that will only match documents with a field that contains exactly this value
*/
public Query exactQuery(Object value, SearchExecutionContext context) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add javadocs?

return new ConstantScoreQuery(termQuery(value, context));
}

/** Build a constant-scoring query that matches all values. The default implementation uses a
* {@link ConstantScoreQuery} around a {@link BooleanQuery} whose {@link Occur#SHOULD} clauses
* are generated with {@link #termQuery}. */
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,11 @@ public Query termQuery(Object value, SearchExecutionContext context) {
return rangeQuery(value, value, true, true, ShapeRelation.INTERSECTS, null, null, context);
}

@Override
public Query exactQuery(Object value, SearchExecutionContext context) {
throw new IllegalArgumentException("Field [" + name() + "] of type [" + typeName() + "] doesn't support exact queries");
}

@Override
public Query rangeQuery(
Object lowerTerm,
Expand Down