Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

require_field_match now defaults to true #11067

Closed
wants to merge 3 commits into from
Closed

require_field_match now defaults to true #11067

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
32d05fe
Highlighting: require_field_match set to true by default
javanna May 8, 2015
175a989
restore search on _all, enabled require_field_match
javanna May 11, 2015
fc17a36
updated docs
javanna May 13, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/reference/migration/migrate_2_0.asciidoc
View file
Open in desktop
Expand Up @@ -508,3 +508,14 @@ Settings settings = ImmutableSettings.settingsBuilder()
.put("cluster.name", "myClusterName").build();
Client client = TransportClient.builder().settings(settings).build();
--------------------------------------------------

=== Highlighting

The default value for the `require_field_match` option is `true` rather than
`false`, meaning that the highlighters will take the fields that were queried
into account by default. That means for instance that highlighting any field
when querying the `_all` field will produce no highlighted snippets by default,
given that the match was on the `_all` field only. Querying the same fields
that need to be highlighted is the cleaner solution to get back highlighted
snippets. Otherwise `require_field_match` option can be set to `false` to
ignore field names completely when highlighting.
8 changes: 4 additions & 4 deletions docs/reference/search/request/highlighting.asciidoc
View file
Open in desktop
Expand Up @@ -406,10 +406,10 @@ at the field level.
[[field-match]]
==== Require Field Match

`require_field_match` can be set to `true` which will cause a field to
be highlighted only if a query matched that field. `false` means that
terms are highlighted on all requested fields regardless if the query
matches specifically on them.
`require_field_match` can be set to `false` which will cause any field to
be highlighted regardless of whether the query matched specifically on them.
The default behaviour is `true`, meaning that only fields that hold a query
match will be highlighted.

[[boundary-characters]]
==== Boundary Characters
Expand Down
2 changes: 1 addition & 1 deletion src/main/java/org/elasticsearch/search/highlight/HighlighterParseElement.java
View file
Open in desktop
Expand Up @@ -82,7 +82,7 @@ public SearchContextHighlight parse(XContentParser parser, IndexQueryParserServi

final SearchContextHighlight.FieldOptions.Builder globalOptionsBuilder = new SearchContextHighlight.FieldOptions.Builder()
.preTags(DEFAULT_PRE_TAGS).postTags(DEFAULT_POST_TAGS).scoreOrdered(false).highlightFilter(false)
.requireFieldMatch(false).forceSource(false).fragmentCharSize(100).numberOfFragments(5)
.requireFieldMatch(true).forceSource(false).fragmentCharSize(100).numberOfFragments(5)
.encoder("default").boundaryMaxScan(SimpleBoundaryScanner.DEFAULT_MAX_SCAN)
.boundaryChars(SimpleBoundaryScanner.DEFAULT_BOUNDARY_CHARS)
.noMatchSize(0).phraseLimit(256);
Expand Down
75 changes: 20 additions & 55 deletions src/test/java/org/elasticsearch/search/highlight/HighlighterSearchTests.java
View file
Open in desktop
Expand Up @@ -21,7 +21,6 @@
import com.carrotsearch.randomizedtesting.generators.RandomPicks;
import com.google.common.base.Joiner;
import com.google.common.collect.Iterables;

import org.apache.lucene.util.LuceneTestCase.Slow;
import org.elasticsearch.Version;
import org.elasticsearch.action.index.IndexRequestBuilder;
Expand All @@ -31,15 +30,9 @@
import org.elasticsearch.common.settings.ImmutableSettings.Builder;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.index.query.BoostableQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.IdsQueryBuilder;
import org.elasticsearch.index.query.MatchQueryBuilder;
import org.elasticsearch.index.query.*;
import org.elasticsearch.index.query.MatchQueryBuilder.Operator;
import org.elasticsearch.index.query.MatchQueryBuilder.Type;
import org.elasticsearch.index.query.MultiMatchQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
Expand All @@ -56,40 +49,12 @@
import static org.elasticsearch.client.Requests.searchRequest;
import static org.elasticsearch.common.settings.ImmutableSettings.settingsBuilder;
import static org.elasticsearch.common.xcontent.XContentFactory.jsonBuilder;
import static org.elasticsearch.index.query.QueryBuilders.missingQuery;
import static org.elasticsearch.index.query.QueryBuilders.typeQuery;
import static org.elasticsearch.index.query.QueryBuilders.boolQuery;
import static org.elasticsearch.index.query.QueryBuilders.boostingQuery;
import static org.elasticsearch.index.query.QueryBuilders.commonTermsQuery;
import static org.elasticsearch.index.query.QueryBuilders.constantScoreQuery;
import static org.elasticsearch.index.query.QueryBuilders.filteredQuery;
import static org.elasticsearch.index.query.QueryBuilders.fuzzyQuery;
import static org.elasticsearch.index.query.QueryBuilders.matchPhrasePrefixQuery;
import static org.elasticsearch.index.query.QueryBuilders.matchPhraseQuery;
import static org.elasticsearch.index.query.QueryBuilders.matchQuery;
import static org.elasticsearch.index.query.QueryBuilders.multiMatchQuery;
import static org.elasticsearch.index.query.QueryBuilders.prefixQuery;
import static org.elasticsearch.index.query.QueryBuilders.queryStringQuery;
import static org.elasticsearch.index.query.QueryBuilders.rangeQuery;
import static org.elasticsearch.index.query.QueryBuilders.regexpQuery;
import static org.elasticsearch.index.query.QueryBuilders.termQuery;
import static org.elasticsearch.index.query.QueryBuilders.wildcardQuery;
import static org.elasticsearch.index.query.QueryBuilders.*;
import static org.elasticsearch.search.builder.SearchSourceBuilder.highlight;
import static org.elasticsearch.search.builder.SearchSourceBuilder.searchSource;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertFailures;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHighlight;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertNoFailures;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertNotHighlighted;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.*;
import static org.elasticsearch.test.hamcrest.RegexMatcher.matches;
import static org.hamcrest.Matchers.anyOf;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasKey;
import static org.hamcrest.Matchers.not;
import static org.hamcrest.Matchers.nullValue;
import static org.hamcrest.Matchers.startsWith;
import static org.hamcrest.Matchers.*;

@Slow
public class HighlighterSearchTests extends ElasticsearchIntegrationTest {
Expand Down Expand Up @@ -493,7 +458,7 @@ public void testHighlightIssue1994() throws Exception {
SearchResponse search = client().prepareSearch()
.setQuery(matchQuery("title", "bug"))
.addHighlightedField("title", -1, 2)
.addHighlightedField("titleTV", -1, 2)
.addHighlightedField("titleTV", -1, 2).setHighlighterRequireFieldMatch(false)
.get();

assertHighlight(search, 0, "title", 0, equalTo("This is a test on the highlighting <em>bug</em> present in elasticsearch"));
Expand Down Expand Up @@ -525,7 +490,7 @@ public void testGlobalHighlightingSettingsOverriddenAtFieldLevel() {
.query(termQuery("field1", "test"))
.highlight(highlight().order("score").preTags("<global>").postTags("</global>").fragmentSize(1).numOfFragments(1)
.field(new HighlightBuilder.Field("field1").numOfFragments(2))
.field(new HighlightBuilder.Field("field2").preTags("<field2>").postTags("</field2>").fragmentSize(50)));
.field(new HighlightBuilder.Field("field2").preTags("<field2>").postTags("</field2>").fragmentSize(50).requireFieldMatch(false)));

SearchResponse searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand All @@ -552,7 +517,7 @@ public void testHighlightingOnWildcardFields() throws Exception {
logger.info("--> highlighting and searching on field*");
SearchSourceBuilder source = searchSource()
.query(termQuery("field-plain", "test"))
.highlight(highlight().field("field*").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field*").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

SearchResponse searchResponse = client().search(searchRequest("test").source(source)).actionGet();

Expand Down Expand Up @@ -638,7 +603,7 @@ public void testPlainHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field1");
source = searchSource()
.query(termQuery("_all", "test"))
.highlight(highlight().field("field1").order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field1").order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand All @@ -647,7 +612,7 @@ public void testPlainHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field2");
source = searchSource()
.query(termQuery("_all", "quick"))
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand All @@ -656,7 +621,7 @@ public void testPlainHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field2");
source = searchSource()
.query(prefixQuery("_all", "qui"))
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand All @@ -665,7 +630,7 @@ public void testPlainHighlighter() throws Exception {
logger.info("--> searching on _all with constant score, highlighting on field2");
source = searchSource()
.query(constantScoreQuery(prefixQuery("_all", "qui")))
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand All @@ -674,7 +639,7 @@ public void testPlainHighlighter() throws Exception {
logger.info("--> searching on _all with constant score, highlighting on field2");
source = searchSource()
.query(boolQuery().should(constantScoreQuery(prefixQuery("_all", "qui"))))
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();
assertHighlight(searchResponse, 0, "field2", 0, 1, equalTo("The <xxx>quick</xxx> brown fox jumps over the lazy dog"));
Expand All @@ -701,7 +666,7 @@ public void testFastVectorHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field1");
source = searchSource()
.query(termQuery("_all", "test"))
.highlight(highlight().field("field1", 100, 0).order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field1", 100, 0).order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand All @@ -711,7 +676,7 @@ public void testFastVectorHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field2");
source = searchSource()
.query(termQuery("_all", "quick"))
.highlight(highlight().field("field2", 100, 0).order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2", 100, 0).order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand All @@ -721,7 +686,7 @@ public void testFastVectorHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field2");
source = searchSource()
.query(prefixQuery("_all", "qui"))
.highlight(highlight().field("field2", 100, 0).order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2", 100, 0).order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().prepareSearch("test").setSource(source.buildAsBytes()).get();

Expand Down Expand Up @@ -1915,7 +1880,7 @@ public void testPostingsHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field1");
source = searchSource()
.query(termQuery("_all", "test"))
.highlight(highlight().field("field1").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field1").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().search(searchRequest("test").source(source)).actionGet();

Expand All @@ -1924,7 +1889,7 @@ public void testPostingsHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field2");
source = searchSource()
.query(termQuery("_all", "quick"))
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2").order("score").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().search(searchRequest("test").source(source)).actionGet();

Expand All @@ -1933,7 +1898,7 @@ public void testPostingsHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field2");
source = searchSource()
.query(matchPhraseQuery("_all", "quick brown"))
.highlight(highlight().field("field2").preTags("<xxx>").postTags("</xxx>"));
.highlight(highlight().field("field2").preTags("<xxx>").postTags("</xxx>").requireFieldMatch(false));

searchResponse = client().search(searchRequest("test").source(source)).actionGet();

Expand All @@ -1944,7 +1909,7 @@ public void testPostingsHighlighter() throws Exception {
logger.info("--> searching on _all, highlighting on field2, falling back to the plain highlighter");
source = searchSource()
.query(matchPhraseQuery("_all", "quick brown"))
.highlight(highlight().field("field2").preTags("<xxx>").postTags("</xxx>").highlighterType("highlighter"));
.highlight(highlight().field("field2").preTags("<xxx>").postTags("</xxx>").highlighterType("highlighter").requireFieldMatch(false));

searchResponse = client().search(searchRequest("test").source(source)).actionGet();

Expand Down Expand Up @@ -1983,7 +1948,7 @@ public void testPostingsHighlighterNumberOfFragments() throws Exception {
.query(termQuery("field1", "fox"))
.highlight(highlight()
.field(new HighlightBuilder.Field("field1").numOfFragments(5).preTags("<field1>").postTags("</field1>"))
.field(new HighlightBuilder.Field("field2").numOfFragments(2).preTags("<field2>").postTags("</field2>")));
.field(new HighlightBuilder.Field("field2").numOfFragments(2).preTags("<field2>").postTags("</field2>").requireFieldMatch(false)));

SearchResponse searchResponse = client().search(searchRequest("test").source(source)).actionGet();

Expand Down