[Issue]: Need to improve the inference documentation to articulate information around adaptive allocations.

[georgewallace]

Type of issue

None

What documentation page is affected

Inference documentation

What happened?

The cost implication of creating a custom inference endpoint without allowing adaptive allocations to scale down to 0 is not clearly explained in our documentation/UI.

Additional info

Additional email will be provided directly.

[georgewallace]

@szabosteve please take a look at this. I will forward you more information, @ppf2 can also provide more information as this was raised in a support case.

[kosabogi]

I started investigating this, and a questions came up:

What exactly do we mean by custom inference endpoints? I initially assumed this referred to all endpoints created for services other than ELSER or elasticsearch - for example, endpoints that use third-party services. However, adaptive allocations don’t appear to be supported for those services. Could you please clarify that what we mean exactly by custom here?

cc @georgewallace @szabosteve @ppf2

[szabosteve]

@kosabogi I think "custom" here means just that it's not a predefined endpoint (provided out-of-the-box), but something that the user themself creates.

[kosabogi]

@kosabogi I think "custom" here means just that it's not a predefined endpoint (provided out-of-the-box), but something that the user themself creates.

I see, thank you for the clarification!