Introducing a new predefined completion endpoint (.gp-llm-v2-completion) #138420
Conversation
elasticsearchmachine
added
needs:triage
| TaskType.COMPLETION, | ||
| ElasticInferenceService.NAME, | ||
| GP_LLM_V2_COMPLETION_SERVICE_SETTINGS, | ||
| ChunkingSettingsBuilder.DEFAULT_SETTINGS |
There was a problem hiding this comment.
I just realized we have a subtle bug here that I introduced initially. I tested and it doesn't seem to affect anything, thankfully. Could you try setting this to null?
There shouldn't be any chunking settings for completion or chat completion endpoints.
There was a problem hiding this comment.
If we could add a test that for these 3 endpoints that the model configuration should not have chunking settings that'd be great 🙏
| TaskType.CHAT_COMPLETION, | ||
| ElasticInferenceService.NAME, | ||
| GP_LLM_V2_COMPLETION_SERVICE_SETTINGS, | ||
| ChunkingSettingsBuilder.DEFAULT_SETTINGS |
There was a problem hiding this comment.
Would you be up for setting this one to null to fix the bug and keep things consistent?
| TaskType.CHAT_COMPLETION, | ||
| ElasticInferenceService.NAME, | ||
| COMPLETION_SERVICE_SETTINGS, | ||
| ChunkingSettingsBuilder.DEFAULT_SETTINGS |
There was a problem hiding this comment.
Would you be up for setting this one to null to fix the bug and keep things consistent?
jonathan-buttner
added
:ml
|
Pinging @elastic/ml-core (Team:ML) |
|
@jonathan-buttner I will do these changes on Monday. |
❤️ thank you! |
3 participants
This PR introduce a new predefined completion endpoint for the
.gp-llm-v2model