Skip to content

[ML] Adding fields for Inference service configuration API #121103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 31, 2025
Merged

[ML] Adding fields for Inference service configuration API #121103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0c79bb9
Adding fields and making some optional
jonathan-buttner Jan 28, 2025
839cfbc
Fixing tests
jonathan-buttner Jan 29, 2025
873cb7f
Merge branch 'main' into ml-inference-missing-config-fields
jonathan-buttner Jan 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .../elasticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -409,7 +409,7 @@ public static InferenceServiceConfiguration get() {
HTTP_SCHEMA_NAME,
new SettingsConfiguration.Builder(supportedTaskTypes).setDescription("")
.setLabel("HTTP Schema")
.setRequired(true)
.setRequired(false)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need to be required because it's defaulted for all task types. For example:

https://github.com/elastic/elasticsearch/blob/main/x-pack/plugin/inference/src/main/java/org/elasticsearch/xpack/inference/external/request/alibabacloudsearch/AlibabaCloudSearchSparseRequest.java#L58

.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.STRING)
Expand Down
15 changes: 15 additions & 0 deletions ...n/java/org/elasticsearch/xpack/inference/services/amazonbedrock/AmazonBedrockService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@
import java.util.Map;
import java.util.Set;

import static org.elasticsearch.xpack.inference.services.ServiceFields.DIMENSIONS;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.createInvalidModelException;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.parsePersistedConfigErrorMsg;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.removeFromMap;
Expand Down Expand Up @@ -413,6 +414,20 @@ public static InferenceServiceConfiguration get() {
.build()
);

configurationMap.put(
DIMENSIONS,
new SettingsConfiguration.Builder(EnumSet.of(TaskType.TEXT_EMBEDDING)).setDescription(
"The number of dimensions the resulting embeddings should have. For more information refer to "
+ "https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-titan-embed-text.html."
)
.setLabel("Dimensions")
.setRequired(false)
.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.INTEGER)
.build()
);

configurationMap.putAll(AmazonBedrockSecretSettings.Configuration.get());
configurationMap.putAll(
RateLimitSettings.toSettingsConfigurationWithDescription(
Expand Down
15 changes: 15 additions & 0 deletions ...n/java/org/elasticsearch/xpack/inference/services/azureaistudio/AzureAiStudioService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
import java.util.Map;
import java.util.Set;

import static org.elasticsearch.xpack.inference.services.ServiceFields.DIMENSIONS;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.createInvalidModelException;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.parsePersistedConfigErrorMsg;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.removeFromMap;
Expand Down Expand Up @@ -441,6 +442,20 @@ public static InferenceServiceConfiguration get() {
.build()
);

configurationMap.put(
DIMENSIONS,
new SettingsConfiguration.Builder(EnumSet.of(TaskType.TEXT_EMBEDDING)).setDescription(
"The number of dimensions the resulting embeddings should have. For more information refer to "
+ "https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-embeddings."
)
.setLabel("Dimensions")
.setRequired(false)
.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.INTEGER)
.build()
);

configurationMap.putAll(DefaultSecretSettings.toSettingsConfiguration(supportedTaskTypes));
configurationMap.putAll(RateLimitSettings.toSettingsConfiguration(supportedTaskTypes));

Expand Down
15 changes: 15 additions & 0 deletions .../main/java/org/elasticsearch/xpack/inference/services/azureopenai/AzureOpenAiService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
import java.util.Map;
import java.util.Set;

import static org.elasticsearch.xpack.inference.services.ServiceFields.DIMENSIONS;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.createInvalidModelException;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.parsePersistedConfigErrorMsg;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.removeFromMap;
Expand Down Expand Up @@ -382,6 +383,20 @@ public static InferenceServiceConfiguration get() {
.build()
);

configurationMap.put(
DIMENSIONS,
new SettingsConfiguration.Builder(EnumSet.of(TaskType.TEXT_EMBEDDING)).setDescription(
"The number of dimensions the resulting embeddings should have. For more information refer to "
+ "https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#request-body-1."
)
.setLabel("Dimensions")
.setRequired(false)
.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.INTEGER)
.build()
);

configurationMap.putAll(AzureOpenAiSecretSettings.Configuration.get());
configurationMap.putAll(
RateLimitSettings.toSettingsConfigurationWithDescription(
Expand Down
15 changes: 15 additions & 0 deletions ...erence/src/main/java/org/elasticsearch/xpack/inference/services/cohere/CohereService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import org.elasticsearch.inference.SettingsConfiguration;
import org.elasticsearch.inference.SimilarityMeasure;
import org.elasticsearch.inference.TaskType;
import org.elasticsearch.inference.configuration.SettingsConfigurationFieldType;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.xpack.inference.chunking.ChunkingSettingsBuilder;
import org.elasticsearch.xpack.inference.chunking.EmbeddingRequestChunker;
Expand All @@ -51,6 +52,7 @@
import java.util.Map;
import java.util.Set;

import static org.elasticsearch.xpack.inference.services.ServiceFields.MODEL_ID;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.createInvalidModelException;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.parsePersistedConfigErrorMsg;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.removeFromMap;
Expand Down Expand Up @@ -363,6 +365,19 @@ public static InferenceServiceConfiguration get() {
() -> {
var configurationMap = new HashMap<String, SettingsConfiguration>();

configurationMap.put(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's optional for the v1 endpoints for cohere: https://docs.cohere.com/v1/reference/embed#request.body.model

It's required for v2 endpoints for cohere. We use v1 right now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

model
string
Required
Defaults to embed-english-v2.0

required but with a default? 🧐

MODEL_ID,
new SettingsConfiguration.Builder(supportedTaskTypes).setDescription(
"The name of the model to use for the inference task."
)
.setLabel("Model ID")
.setRequired(false)
.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.STRING)
.build()
);

configurationMap.putAll(DefaultSecretSettings.toSettingsConfiguration(supportedTaskTypes));
configurationMap.putAll(RateLimitSettings.toSettingsConfiguration(supportedTaskTypes));

Expand Down
2 changes: 2 additions & 0 deletions ...java/org/elasticsearch/xpack/inference/services/googlevertexai/GoogleVertexAiService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,8 @@ public static InferenceServiceConfiguration get() {
() -> {
var configurationMap = new HashMap<String, SettingsConfiguration>();

// TODO whether the model ID is required or not depends on the task type
// For rerank it is optional, for text_embedding it is required
configurationMap.put(
MODEL_ID,
new SettingsConfiguration.Builder(supportedTaskTypes).setDescription("ID of the LLM you're using.")
Expand Down
29 changes: 29 additions & 0 deletions ...erence/src/main/java/org/elasticsearch/xpack/inference/services/jinaai/JinaAIService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import org.elasticsearch.inference.SettingsConfiguration;
import org.elasticsearch.inference.SimilarityMeasure;
import org.elasticsearch.inference.TaskType;
import org.elasticsearch.inference.configuration.SettingsConfigurationFieldType;
import org.elasticsearch.rest.RestStatus;
import org.elasticsearch.xpack.inference.chunking.ChunkingSettingsBuilder;
import org.elasticsearch.xpack.inference.chunking.EmbeddingRequestChunker;
Expand All @@ -49,6 +50,7 @@
import java.util.List;
import java.util.Map;

import static org.elasticsearch.xpack.inference.services.ServiceFields.DIMENSIONS;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.createInvalidModelException;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.parsePersistedConfigErrorMsg;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.removeFromMap;
Expand Down Expand Up @@ -339,6 +341,33 @@ public static InferenceServiceConfiguration get() {
() -> {
var configurationMap = new HashMap<String, SettingsConfiguration>();

configurationMap.put(
JinaAIServiceSettings.MODEL_ID,
new SettingsConfiguration.Builder(supportedTaskTypes).setDescription(
"The name of the model to use for the inference task."
)
.setLabel("Model ID")
.setRequired(true)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Required for both text embedding and rerank: https://api.jina.ai/redoc#tag/embeddings

.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.STRING)
.build()
);

configurationMap.put(
DIMENSIONS,
new SettingsConfiguration.Builder(EnumSet.of(TaskType.TEXT_EMBEDDING)).setDescription(
"The number of dimensions the resulting embeddings should have. For more information refer to "
+ "https://api.jina.ai/redoc#tag/embeddings/operation/create_embedding_v1_embeddings_post."
)
.setLabel("Dimensions")
.setRequired(false)
.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.INTEGER)
.build()
);

configurationMap.putAll(DefaultSecretSettings.toSettingsConfiguration(supportedTaskTypes));
configurationMap.putAll(RateLimitSettings.toSettingsConfiguration(supportedTaskTypes));

Expand Down
19 changes: 8 additions & 11 deletions ...erence/src/main/java/org/elasticsearch/xpack/inference/services/openai/OpenAiService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@

import static org.elasticsearch.xpack.inference.external.action.ActionUtils.constructFailedToSendRequestMessage;
import static org.elasticsearch.xpack.inference.external.action.openai.OpenAiActionCreator.COMPLETION_ERROR_PREFIX;
import static org.elasticsearch.xpack.inference.services.ServiceFields.DIMENSIONS;
import static org.elasticsearch.xpack.inference.services.ServiceFields.MODEL_ID;
import static org.elasticsearch.xpack.inference.services.ServiceFields.URL;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.createInvalidModelException;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.parsePersistedConfigErrorMsg;
import static org.elasticsearch.xpack.inference.services.ServiceUtils.removeFromMap;
Expand Down Expand Up @@ -440,19 +440,16 @@ public static InferenceServiceConfiguration get() {
);

configurationMap.put(
URL,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to expose the URL for users. This was meant to be an internal way to test. Although I know we've leveraged that to connect to other providers that adhere to the OpenAI spec. So If we want to keep this I can understand the argument for that. We should make it optional though. Let me know what you think.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

URL is configurable in the API so that it can be used with a LocalAI server https://www.elastic.co/search-labs/blog/localai-for-text-embeddings

It shouldn't be configurable in the UI though as it makes little sense to change this when creating up an OpenAI endpoint

new SettingsConfiguration.Builder(SUPPORTED_TASK_TYPES_FOR_SERVICES_API).setDefaultValue(
"https://api.openai.com/v1/chat/completions"
DIMENSIONS,
new SettingsConfiguration.Builder(EnumSet.of(TaskType.TEXT_EMBEDDING)).setDescription(
"The number of dimensions the resulting embeddings should have. For more information refer to "
+ "https://platform.openai.com/docs/api-reference/embeddings/create#embeddings-create-dimensions."
)
.setDescription(
"The OpenAI API endpoint URL. For more information on the URL, refer to the "
+ "https://platform.openai.com/docs/api-reference."
)
.setLabel("URL")
.setRequired(true)
.setLabel("Dimensions")
.setRequired(false)
.setSensitive(false)
.setUpdatable(false)
.setType(SettingsConfigurationFieldType.STRING)
.setType(SettingsConfigurationFieldType.INTEGER)
.build()
);

Expand Down
2 changes: 1 addition & 1 deletion ...ticsearch/xpack/inference/services/alibabacloudsearch/AlibabaCloudSearchServiceTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -490,7 +490,7 @@ public void testGetConfiguration() throws Exception {
"http_schema": {
"description": "",
"label": "HTTP Schema",
"required": true,
"required": false,
"sensitive": false,
"updatable": false,
"type": "str",
Expand Down
Loading