New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ML] add new trained_models/{model_id}/_infer endpoint for all supervised models and deprecate deployment infer api #86361
Merged
benwtrent
merged 15 commits into
elastic:master
from
benwtrent:feature/ml-single-infer-endpoint
May 5, 2022
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
9017889
[ML] add new trained_models/{model_id}/_infer endpoint for all superv…
benwtrent dd96fc3
Update docs/changelog/86361.yaml
benwtrent 5e50f71
fixing test
benwtrent 8b1db2c
Merge branch 'feature/ml-single-infer-endpoint' of github.com:benwtre…
benwtrent 613a49a
fixing format
benwtrent 2fd8ac9
fixing format
benwtrent a54065a
[DOCS] Re-add deprecated API
lcawl 9b51c1c
Update 86361.yaml
benwtrent 4057c3d
fixing spec
benwtrent 82829f4
Merge branch 'master' into feature/ml-single-infer-endpoint
elasticmachine e8714e1
Merge branch 'master' into feature/ml-single-infer-endpoint
benwtrent f06c4a0
Merge branch 'feature/ml-single-infer-endpoint' of github.com:benwtre…
benwtrent 3f1fee0
adding tests
benwtrent 0e51d67
cleaning up test
benwtrent 7492678
fixing tests
benwtrent File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
pr: 86361 | ||
summary: "Add new _infer endpoint for all supervised models and deprecate deployment infer api" | ||
area: Machine Learning | ||
type: enhancement | ||
issues: [] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
271 changes: 271 additions & 0 deletions
271
docs/reference/ml/trained-models/apis/infer-trained-model.asciidoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,271 @@ | ||
[role="xpack"] | ||
[[infer-trained-model]] | ||
= Infer trained model API | ||
[subs="attributes"] | ||
++++ | ||
<titleabbrev>Infer trained model</titleabbrev> | ||
++++ | ||
|
||
Evaluates a trained model. The model may be any supervised model either trained by {dfanalytics} or imported. | ||
|
||
[[infer-trained-model-request]] | ||
== {api-request-title} | ||
|
||
`POST _ml/trained_models/<model_id>/_infer` | ||
|
||
//// | ||
[[infer-trained-model-prereq]] | ||
== {api-prereq-title} | ||
|
||
//// | ||
//// | ||
[[infer-trained-model-desc]] | ||
== {api-description-title} | ||
|
||
//// | ||
|
||
[[infer-trained-model-path-params]] | ||
== {api-path-parms-title} | ||
|
||
`<model_id>`:: | ||
(Required, string) | ||
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=model-id] | ||
|
||
[[infer-trained-model-query-params]] | ||
== {api-query-parms-title} | ||
|
||
`timeout`:: | ||
(Optional, time) | ||
Controls the amount of time to wait for {infer} results. Defaults to 10 seconds. | ||
|
||
[[infer-trained-model-request-body]] | ||
== {api-request-body-title} | ||
|
||
`docs`:: | ||
(Required, array) | ||
An array of objects to pass to the model for inference. The objects should | ||
contain the fields matching your configured trained model input. Typically for NLP models, the field | ||
name is `text_field`. Currently for NLP models, only a single value is allowed. For {dfanalytics} or | ||
imported classification or regression models, more than one value is allowed. | ||
|
||
//// | ||
[[infer-trained-model-results]] | ||
== {api-response-body-title} | ||
//// | ||
//// | ||
[[ml-get-trained-models-response-codes]] | ||
== {api-response-codes-title} | ||
|
||
//// | ||
|
||
[[infer-trained-model-example]] | ||
== {api-examples-title} | ||
|
||
The response depends on the kind of model. | ||
|
||
For example, for language identification the response is the predicted language and the score: | ||
|
||
[source,console] | ||
-------------------------------------------------- | ||
POST _ml/trained_models/lang_ident_model_1/_infer | ||
{ | ||
"docs":[{"text": "The fool doth think he is wise, but the wise man knows himself to be a fool."}] | ||
} | ||
-------------------------------------------------- | ||
// TEST[skip:TBD] | ||
|
||
Here are the results predicting english with a high probability. | ||
|
||
[source,console-result] | ||
---- | ||
{ | ||
"inference_results": [ | ||
{ | ||
"predicted_value": "en", | ||
"prediction_probability": 0.9999658805366392, | ||
"prediction_score": 0.9999658805366392 | ||
} | ||
] | ||
} | ||
---- | ||
// NOTCONSOLE | ||
|
||
|
||
When it is a text classification model, the response is the score and predicted classification. | ||
|
||
For example: | ||
|
||
[source,console] | ||
-------------------------------------------------- | ||
POST _ml/trained_models/model2/_infer | ||
{ | ||
"docs": [{"text_field": "The movie was awesome!!"}] | ||
} | ||
-------------------------------------------------- | ||
// TEST[skip:TBD] | ||
|
||
The API returns the predicted label and the confidence. | ||
|
||
[source,console-result] | ||
---- | ||
{ | ||
"inference_results": [{ | ||
"predicted_value" : "POSITIVE", | ||
"prediction_probability" : 0.9998667964092964 | ||
}] | ||
} | ||
---- | ||
// NOTCONSOLE | ||
|
||
For named entity recognition (NER) models, the response contains the annotated | ||
text output and the recognized entities. | ||
|
||
[source,console] | ||
-------------------------------------------------- | ||
POST _ml/trained_models/model2/_infer | ||
{ | ||
"docs": [{"text_field": "Hi my name is Josh and I live in Berlin"}] | ||
} | ||
-------------------------------------------------- | ||
// TEST[skip:TBD] | ||
|
||
The API returns in this case: | ||
|
||
[source,console-result] | ||
---- | ||
{ | ||
"inference_results": [{ | ||
"predicted_value" : "Hi my name is [Josh](PER&Josh) and I live in [Berlin](LOC&Berlin)", | ||
"entities" : [ | ||
{ | ||
"entity" : "Josh", | ||
"class_name" : "PER", | ||
"class_probability" : 0.9977303419824, | ||
"start_pos" : 14, | ||
"end_pos" : 18 | ||
}, | ||
{ | ||
"entity" : "Berlin", | ||
"class_name" : "LOC", | ||
"class_probability" : 0.9992474323902818, | ||
"start_pos" : 33, | ||
"end_pos" : 39 | ||
} | ||
] | ||
}] | ||
} | ||
---- | ||
// NOTCONSOLE | ||
|
||
Zero-shot classification models require extra configuration defining the class labels. | ||
These labels are passed in the zero-shot inference config. | ||
|
||
[source,console] | ||
-------------------------------------------------- | ||
POST _ml/trained_models/model2/_infer | ||
{ | ||
"docs": [ | ||
{ | ||
"text_field": "This is a very happy person" | ||
} | ||
], | ||
"inference_config": { | ||
"zero_shot_classification": { | ||
"labels": [ | ||
"glad", | ||
"sad", | ||
"bad", | ||
"rad" | ||
], | ||
"multi_label": false | ||
} | ||
} | ||
} | ||
-------------------------------------------------- | ||
// TEST[skip:TBD] | ||
|
||
The API returns the predicted label and the confidence, as well as the top classes: | ||
|
||
[source,console-result] | ||
---- | ||
{ | ||
"inference_results": [{ | ||
"predicted_value" : "glad", | ||
"top_classes" : [ | ||
{ | ||
"class_name" : "glad", | ||
"class_probability" : 0.8061155063386439, | ||
"class_score" : 0.8061155063386439 | ||
}, | ||
{ | ||
"class_name" : "rad", | ||
"class_probability" : 0.18218006158387956, | ||
"class_score" : 0.18218006158387956 | ||
}, | ||
{ | ||
"class_name" : "bad", | ||
"class_probability" : 0.006325615787634201, | ||
"class_score" : 0.006325615787634201 | ||
}, | ||
{ | ||
"class_name" : "sad", | ||
"class_probability" : 0.0053788162898424545, | ||
"class_score" : 0.0053788162898424545 | ||
} | ||
], | ||
"prediction_probability" : 0.8061155063386439 | ||
}] | ||
} | ||
---- | ||
// NOTCONSOLE | ||
|
||
|
||
The tokenization truncate option can be overridden when calling the API: | ||
|
||
[source,console] | ||
-------------------------------------------------- | ||
POST _ml/trained_models/model2/_infer | ||
{ | ||
"docs": [{"text_field": "The Amazon rainforest covers most of the Amazon basin in South America"}], | ||
"inference_config": { | ||
"ner": { | ||
"tokenization": { | ||
"bert": { | ||
"truncate": "first" | ||
} | ||
} | ||
} | ||
} | ||
} | ||
-------------------------------------------------- | ||
// TEST[skip:TBD] | ||
|
||
When the input has been truncated due to the limit imposed by the model's `max_sequence_length` | ||
the `is_truncated` field appears in the response. | ||
|
||
[source,console-result] | ||
---- | ||
{ | ||
"inference_results": [{ | ||
"predicted_value" : "The [Amazon](LOC&Amazon) rainforest covers most of the [Amazon](LOC&Amazon) basin in [South America](LOC&South+America)", | ||
"entities" : [ | ||
{ | ||
"entity" : "Amazon", | ||
"class_name" : "LOC", | ||
"class_probability" : 0.9505460915724254, | ||
"start_pos" : 4, | ||
"end_pos" : 10 | ||
}, | ||
{ | ||
"entity" : "Amazon", | ||
"class_name" : "LOC", | ||
"class_probability" : 0.9969992804311777, | ||
"start_pos" : 41, | ||
"end_pos" : 47 | ||
} | ||
], | ||
"is_truncated" : true | ||
}] | ||
} | ||
---- | ||
// NOTCONSOLE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add an example using the built in
lang_ident
model pls