Search API: Support highlighting #69

kimchy · 2010-03-18T14:02:49Z

Allow to highlight search results on one or more fields. The implementation uses the lucene fast-vector-highlighter. The search request body:

{
    query : {...},
    highlight : {
        fields : {
            "_all" : {}
        }
    }
}

In the above case, the _all field will be highlighted for each search hit (there will be another element in each search each, called highlight, which includes the highlighted fields and the highlighted fragments).

Note, in order to highlight, the field in question must be stored and have termVector of with_positions_offsets. In the above case, the mapping should include:

{
    typeName : {
        allField : {store : "yes", termVector : "with_positions_offsets"}
    }
}

The _all field can be an easy to use candidate for highlighting.

By default, the highlighting will wrap highlighted text in <em> and </em>. This can be controlled by setting preTags and postTags, for example:

{
    query : {...},
    highlight : {
        preTags : ["<tag1>", "<tag2>"],
        postTags : ["</tag1>", "</tag2>"]
        fields : {
            "_all" : {}
        }
    }
}

There can be a single tag or more, and the "importance" is ordered. There are also built in "tag" schemas, with currently a single schema called styled with preTags of:

<em class="hlt1">, <em class="hlt2">, <em class="hlt2">,
<em class="hlt3">, <em class="hlt4">, <em class="hlt5">,
<em class="hlt6">, <em class="hlt7">, <em class="hlt8">,
<em class="hlt9">

And post tag of </em>. If you think of more nice to have built in tag schemas, just commend on this issue or post another issue to create it. Here is an example of switching tag schemas:

{
    query : {...},
    highlight : {
        tagSchema : "styled",
        fields : {
            "_all" : {}
        }
    }
}

Each field highlighted can control the size of the highlighted fragment in characters (defaults to 100), and the maximum number of fragments to return (defaults to 5). For example:

{
    query : {...},
    highlight : {
        fields : {
            "_all" : {fragmentSize : 150, numberOfFragments : 3}
        }
    }
}

The text was updated successfully, but these errors were encountered:

kimchy · 2010-03-18T14:05:35Z

Search API: Support highlighting, closed by 28b0b5f.

phoet · 2012-04-10T14:11:03Z

is the _all field a special document attribute, or some other kind of special field that one can add to a document? is there further info on this?

kimchy · 2012-04-15T12:30:11Z

@phoet see http://www.elasticsearch.org/guide/reference/mapping/all-field.html, questions are best asked on the mailing list, since issues are not monitored by the whole ES community.

With elastic#68 we replaced `language`field with `_language`. We can now remove the old deprecated name. Closes elastic#69. (cherry picked from commit e39f144)

This commit updates JodaTime to version 2.9.5 that contain a fix for a bug when parsing time zones (see JodaOrg/joda-time#332, JodaOrg/joda-time#386 and JodaOrg/joda-time#373). It also remove the joda-convert dependency that seems to be unused. closes #20911 Here is the changelog for 2.9.5: ``` Changes in 2.9.5 ---------------- - Add Norwegian period translations [#378] - Add Duration.dividedBy(long,RoundingMode) [#69, #379] - DateTimeZone data updated to version 2016i - Fixed bug where clock read twice when comparing two nulls in DateTimeComparator [#404] - Fixed minor issues with historic time-zone data [#373] - Fix bug in time-zone binary search [#332, #386] The fix in v2.9.2 caused problems when the time-zone being parsed was not the last element in the input string. New approach uses a different approach to the problem. - Update tests for JDK 9 [#394] - Close buffered reader correctly in zone info compiler [#396] - Handle locale correctly zone info compiler [#397] ```

This commit updates JodaTime to version 2.9.5 that contain a fix for a bug when parsing time zones (see JodaOrg/joda-time#332, JodaOrg/joda-time#386 and JodaOrg/joda-time#373). It also remove the joda-convert dependency that seems to be unused. closes #20911 Here is the changelog for 2.9.5: ``` Changes in 2.9.5 ---------------- - Add Norwegian period translations [#378] - Add Duration.dividedBy(long,RoundingMode) [#69, #379] - DateTimeZone data updated to version 2016i - Fixed bug where clock read twice when comparing two nulls in DateTimeComparator [#404] - Fixed minor issues with historic time-zone data [#373] - Fix bug in time-zone binary search [#332, #386] The fix in v2.9.2 caused problems when the time-zone being parsed was not the last element in the input string. New approach uses a different approach to the problem. - Update tests for JDK 9 [#394] - Close buffered reader correctly in zone info compiler [#396] - Handle locale correctly zone info compiler [#397] ``` (cherry picked from commit 2e53190)

Fixes elastic#69

add id extraction add id awareness to document writes (create/index) add M&R test relates to elastic#69

relates elastic#69

remove serialization from buffered rest client introduce Command classes as a templating mechanism encapsulating underlying serialization relates elastic#69

relates to elastic#69

relates elastic#69

relates to elastic#69

add integration tests reset ES settings within Hadoop configuration relates to elastic#69

relates to elastic#69

relates to elastic#66 elastic#85 elastic#81 elastic#69

refactor command infrastructure add internal command templating improved serialization limiting 'garbage' copies relates to elastic#66 elastic#85 elastic#81 elastic#69

relates to elastic#66 elastic#85 elastic#81 elastic#69

…lastic#69) Update lambda scale to handle all instance types for terraform stack * This commit is for https://engageli.atlassian.net/browse/MPC-1455 It removes the restriction on the scaling lambda, so it will now scale all instances in the terraform stack, except for RDS / REDIS. The instances to be scaled, are now determined by the instance roles passed via the event JSON for the lambda, and these roles must match the tag:Role for each instance. Additionally, it changes the order of operation so that it will scale the standard EC2 instances first, and then perform the instance refresh for the auto-scaling group `ams`. This was required since if an auto-scaling event was already in progress when the lamdba executed, it did not correctly process the standard EC2 instances. modified: aws/ams-cluster-v1-tf/scale.tf modified: aws/scale-lambda-source-code/README modified: aws/scale-lambda-source-code/sample-event.json modified: aws/scale-lambda-source-code/scale.py * fixup modified: aws/scale-lambda-source-code/scale.py Approved-by: Gideon Avida

With this commit we set `JAVA[7-11]_HOME` before starting the Rally daemon so Rally can use those to resolve the proper path to the respective JDK. Relates elastic/rally#518 Closes elastic#69 Relates elastic#72

bluelu mentioned this issue Dec 11, 2014

Node is not responsive after the end of a big merge for close to 10 minutes #8905

Closed

makeyang mentioned this issue Sep 7, 2015

es 0.90.2 plus jdk6.0_25-b06 crashed on production #13368

Closed

rmuir pushed a commit to rmuir/elasticsearch that referenced this issue Nov 8, 2015

Remove deprecated language forced field

a3bb103

With elastic#68 we replaced `language`field with `_language`. We can now remove the old deprecated name. Closes elastic#69. (cherry picked from commit e39f144)

tlrx mentioned this issue Nov 10, 2016

Update Joda Time to version 2.9.5 #21468

Merged

sagar-rout mentioned this issue May 25, 2018

Unable to create index using java high level rest client #30873

Closed

tmortensen mentioned this issue Aug 21, 2018

index setting number_of_routing_shards does not show up in settings of newly created index. But you can split an index if you set this value. #33036

Open

ClaudioMFreitas pushed a commit to ClaudioMFreitas/elasticsearch-1 that referenced this issue Nov 12, 2019

Merged debian init script changes from elastic#16718.

b5ed2fd

Fixes elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add create/index options

ed84d86

add id extraction add id awareness to document writes (create/index) add M&R test relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add new MR API test code

7506776

relates elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

remove unused code (yay!)

cdb8185

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add more checks regarding write operation and id

1d0b1b9

relates elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

minor configuration fix

e780a4e

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add proper prepare() to update command

6438738

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

improve RestClient to parse bulk responses for errors

398ef9c

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

update MR tests (the old API ones)

30acd93

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

move "doc_as_update" to the front of the command (eases debugging)

61fa5e0

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

improve MR old api tests

af824fe

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

update MR tests (new api)

575ab9d

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add more read tests to MR new API suite

d62d46e

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

clean-up MR old api test

4fcdfae

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

minor rename of operation

e8cee57

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add new commands infrastructure to Hive

e375e60

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

fix index for MR new API tests

d7499e9

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

first take at adding Hive support

fd477dd

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

finish Hive integration

819f5d3

add integration tests reset ES settings within Hadoop configuration relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add id awareness to Pig

97ac095

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

remove compiler warning

0096fa8

relates to elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

initial draft of parent_child support

1333d53

relates to elastic#66 elastic#85 elastic#81 elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

align Hive writer with new command serialization

1bdbdc5

relates to elastic#66 elastic#85 elastic#81 elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

add missing class

45e95ab

relates to elastic#66 elastic#85 elastic#81 elastic#69

williamrandolph pushed a commit to williamrandolph/elasticsearch that referenced this issue Jun 4, 2020

fix data leak BRC when dealing with Hive data

56fde61

relates to elastic#66 elastic#85 elastic#81 elastic#69

This issue was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Search API: Support highlighting #69

Search API: Support highlighting #69

kimchy commented Mar 18, 2010

kimchy commented Mar 18, 2010

phoet commented Apr 10, 2012

kimchy commented Apr 15, 2012

Search API: Support highlighting #69

Search API: Support highlighting #69

Comments

kimchy commented Mar 18, 2010

kimchy commented Mar 18, 2010

phoet commented Apr 10, 2012

kimchy commented Apr 15, 2012