Skip to content

Adds new parameters to the elasticsearch inference API for the rerank task type #5476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Oct 20, 2025
Merged

Adds new parameters to the elasticsearch inference API for the rerank task type #5476

merged 6 commits into from
Oct 20, 2025

Conversation

@kosabogi
Copy link
Contributor

@kosabogi kosabogi commented Oct 13, 2025 •
edited
Loading

This PR adds the new long_document_strategy and max_chunks_per_doc parameters to the service_settings object of the Create an Elasticsearch inference endpoint documentation.

It also updates the description of the chunking_settings object to clarify that this setting is only applicable for the sparse_embeddings and text_embeddings task types.

Related issue: #5451

@github-actions
Copy link
Contributor

github-actions bot commented Oct 13, 2025 •
edited
Loading

Following you can find the validation changes against the target branch for the APIs.

API Status Request Response
index 🟢 1445/1445 → 1443/1443 1447/1447 → 1445/1445
indices.create 🔴 1378/1402 → 1385/1409 1402/1402 → 1409/1409
indices.refresh 🟢 329/329 → 327/327 329/329 → 327/327
ml.get_job_stats 🟢 30/30 → 29/29 30/30 → 29/29
ml.put_job 🟢 65/65 → 64/64 65/65 → 64/64

You can validate these APIs yourself by using the make validate target.

@pquentin pquentin changed the title [9.2] Adds new parameters to the elasticsearch inference API for the rerank task type Adds new parameters to the elasticsearch inference API for the rerank task type Oct 14, 2025
dan-rubinstein
dan-rubinstein reviewed Oct 15, 2025
View reviewed changes
specification/inference/_types/CommonTypes.ts Outdated
*/
num_threads: integer
/**
* Only for the `rerank` task type.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A quick clarification. For 9.2, these two values are only configurable for rerank endpoints using the elastic reranker model.

specification/inference/put_elasticsearch/PutElasticsearchRequest.ts Outdated
body: {
/**
* The chunking configuration object.
* The chunking configuration object. For the `rerank` task type, you can enable chunking by setting the `long_document_strategy` parameter to `chunk` in the `service_settings` object.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we need to be more specific about this anywhere but for this new method of chunking the user can not set chunking_settings the way that they would for embeddings. We handle building the chunking settings for them. If we want to clarify how we build the chunking settings somewhere we can.

specification/inference/_types/CommonTypes.ts
*
* Possible values:
* - `truncate` (default): Processes only the beginning of each document.
* - `chunk`: Splits long documents into smaller parts (chunks) before inference.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure where it's best to clarify this but with chunking enabled we will return to the user a single score per document (same as we do for truncating) with the score correlating to the highest score of any chunk. I just want to make it clear that the structure of the response to the user will not change, only the rerank relevance scores.

@kosabogi kosabogi requested a review from davidkyle October 16, 2025 13:28
davidkyle
davidkyle approved these changes Oct 16, 2025
View reviewed changes
Copy link
Member

@davidkyle davidkyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kosabogi kosabogi merged commit a53ef07 into main Oct 20, 2025
8 checks passed
@kosabogi kosabogi deleted the new-params-inference branch October 20, 2025 11:37
github-actions bot pushed a commit that referenced this pull request Oct 20, 2025
@kosabogi @github-actions
Adds new parameters to the elasticsearch inference API for the rerank…
2856899 
… task type (#5476)

* Adds new parameters to the elasticsearch inference rerank API

* Adds unique inference chunking settings for elasticsearch

* Addresses suggestions

(cherry picked from commit a53ef07)
@github-actions github-actions bot mentioned this pull request Oct 20, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dan-rubinstein dan-rubinstein dan-rubinstein left review comments

@davidkyle davidkyle davidkyle approved these changes

@leemthompo leemthompo leemthompo approved these changes

Assignees

No one assigned

Labels

backport 9.2 specification

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kosabogi @davidkyle @leemthompo @dan-rubinstein