[Bug]: KServe microcopy

[lucferbux]

Is there an existing issue for this?

• I have searched the existing issues

Deploy type

OpenDataHub core version (eg. v1.6.0)

Version

2.5

Issues

Replace all "Single model" references with "Single-model" for KServe

Right now, all the references for KServe are displayed as "Single model", but all the ModelMesh references are marked as "Multi-model" with a hyphen.

All the references for KServe to be named "Single-model" with a hyphen.

1. Go to the Custom Serving Runtimes section
2. If you have kserve enable, the tag should display "Single model"
3. If you create a new model, you'll get "Single model"

Issues in Custom Serving Runtimes Dropdown in Create view

In Custom Serving Runtimes, when adding/editing a new SR, we get this dropdown:

1. Change Both single and multi-model serving platforms for Single-model and multi-model serving platforms

2. Change the width of the dropdown to be as wide as the longest label

3. Go to Custom Serving Runtime

4. Create a new Serving Runtime

5. Check the dropdown

Typo in Serving Runtime platform settings

Right no this is the description of the settings section for serving platform.

The correct label should display:

Select the serving platforms that can be used for deploying models on this cluster.

1. Go to Settings > Cluster settings
2. Check the Model Serving platforms section

Typo in Custom Serving Runtimes subheader

This is the current subheader

It should say:

Manage your model serving runtimes

1. Go to Custom Serving Runtimes

Wrong label in deletion modal for model serving

Right now, the deletion modal displays the following message:

It should say:

Type <name> to confirm deletion.

1. Create a new kserve model in either projects or global view
2. Delete it

Wrong labels in platform selection view

These are the current labels for the platform selection view

• Model server cards descriptions (replace “from” with “to”): “Each model is deployed on…” “Multiple models can be deployed on…”
• Sentence revision: “This platform works well for large models or models that need dedicated resources.” for Single model serving platform
• Sentence revision: “This platform works well for sharing resources amongst deployed models.” For multi model serving platform

1. Have both Single model and multi model selected in your cluster
2. Navigate to a new project
3. Check the labels

Inconsistent empty state for kserve in global view

We currently have this empty view for a KServe Project

To have the following label:

To get started, deploy a model

1. Enable only kserve
2. Go to Model Serving view
3. Select an empty project

[bdattoma]

@lucferbux what about changing the text which appear when user disables model serving platforms?

I feel it's difficult to understand the difference between the 2 platforms from these messages:

• too many words
• the two messages are too similar
• the message explains the opposite mode instead of the one which just got disabled - I can see the logic behind the choice, can make sense. But in the complex I find it a bit confusing
• the fact that the messages appear only after disabling an option

Moreover I can see users saying "oh, I want to deploy multiple models, not only one! so I select multi-model" - I think a such choice should be helped in UI rather than requiring to read docs, which are good for getting more details if they want

I would suggest the following changes:

1. describe the 2 options, maybe under a tooltip (? icon)
2. reduce the text
3. when users disable one of the option, a short message appears informing that the new setting is going to affect only new projects
4. Maybe adding example use cases for both, like Single-model, recommended for LargeLanguageModels

Wdyt?

[Xaenalt]

This is probably coming a bit late in the dev cycle, but given the confusion expressed on the call: is there a compelling reason to hide the technology names? My vote would be to call them KServe and ModelMesh tbh

[vconzola]

@Xaenalt Hiding the technology names was part of the requirements for KServe from the BU.

[Xaenalt]

Ok, bit odd, but yeah on reflection on the copy overall, it does seem a bit confusing to the user, I think most users would probably read the titles and select multi-model serving just based on the info there, and since KServe is the overall future direction, we may want to update the language to prefer KServe somehow

[vconzola]

@Xaenalt For new installs KServe=ON, MM=OFF is the default selection. That should show preference for KServe. Also, this is an admin UI. Presumably, anyone working in it should have some idea of what they're doing. That said, we can improve the microcopy of the alerts to improve their readability. We can also add some additional text below each radio button option to better explain what the difference is. We'll likely add something similar to what we show on the platform selection cards in Models and Model Servers when both single and multi serving are enabled (see below).

[Xaenalt]

Maybe even Advanced Model Serving or Basic Model Serving or something, given the confusion with single/multi language, idk

[israel-hdez]

I can see that the "multi-model" and "single-model" wording can be confusing. I do think the most important point that can lead to mistakes is the one that @bdattoma mentions:

Moreover I can see users saying "oh, I want to deploy multiple models, not only one! so I select multi-model" - I think a such choice should be helped in UI rather than requiring to read docs, which are good for getting more details if they want

Like @Xaenalt pointed, I'm not sure if it is already too late to change terminology. Given the descriptions I can read in the shared images (specifically, this one), if there is still a chance to change terminology, I'd suggest to change:

• Multi-model serving platform -> Shared model server orchestrator
• Single-model serving platform -> Dedicated model server orchestrator

...or something like that... trying to leave out the "platform" word, assuming ODS is the platform.

IMHO, the shared/dedicated terminology will reflect with more accuracy what we are trying to communicate in the UI.

[andrewballantyne]

@kaedward is out this week. So this may need to delay for the next release of KServe instead of the first one we are running at.

cc @dgutride @lucferbux

[kaedward]

@vconzola @andrewballantyne @lucferbux

Lots of potential changes! I'll try to address everything mentioned here, but LMK if I miss anything or have gotten anything mixed up...
In these suggestions, I’m also taking into account the conversation that took place on slack. This is a lot of info, and I'm finding it difficult to cut length without losing key points that users might need.

1. Model serving platform names
@israel-hdez If this change is more clear and just as accurate, I don't see a problem with it. But we will need to update this terminology everywhere. I'm also not sure if it's too late to change this, though.
Multi-model serving platform -> Shared model server orchestrator
Single-model serving platform -> Dedicated model server orchestrator

2. Model serving platform descriptions
Single model serving platform:
Each model is deployed from its own model server. Only use this option for large language models that will be deployed using the Caikit runtime.

Multi-model:
Multiple models are deployed from the same model server. Use this option when deploying small models that can share server resources.

3. Disabling model serving platforms info alerts
Single is disabled:
Disabling dedicated model serving prevents large language models from being deployed from projects on this cluster. You will only be able to deploy smaller models that can share a model server. Existing projects with deployed models will continue using their selected serving orchestrator.

Multi is disabled: Disabling shared model serving prevents models from being deployed from shared model servers. Instead, they will use dedicated model servers. Existing projects with deployed models will continue using their selected serving orchestrator.

[Xaenalt]

Yeah, upon reading it, I like the shared/dedicated model server orchestrator terminology

[vconzola]

I'm ok with "shared" and "dedicated" but the term "platform" (vs. orchestrator) comes directly from the KServe Github page - "KServe is a standard, cloud agnostic Model Inference Platform on Kubernetes, built for highly scalable use cases." @kaedward I think you have the final say here.

[israel-hdez]

the term "platform" (vs. orchestrator) comes directly from the KServe

Well... we already have platforms inside platforms 😉
Use what fits best for the cards :-)

[kaedward]

@vconzola thanks! Let's keep "platform" then :-)
Updated suggestions:

1. Model serving platform names
   Multi-model serving platform -> Shared model server platform
   Single-model serving platform -> Dedicated model server platform

2. Model serving platform descriptions
   Single model serving platform:
   Each model is deployed from its own model server. Only use this option for large language models that will be deployed using the Caikit runtime.

Multi-model:
Multiple models are deployed from the same model server. Use this option when deploying small models that can share server resources.

3. Disabling model serving platforms info alerts
   Single is disabled:
   Disabling dedicated model serving prevents large language models from being deployed from projects on this cluster. You will only be able to deploy smaller models that can share a model server. Existing projects with deployed models will continue using their selected serving platform.

Multi is disabled: Disabling shared model serving prevents models from being deployed from shared model servers. Instead, they will use dedicated model servers. Existing projects with deployed models will continue using their selected serving platform.

[bdattoma]

@vconzola thanks! Let's keep "platform" then :-) Updated suggestions:

1. Model serving platform names
   Multi-model serving platform -> Shared model server platform
   Single-model serving platform -> Dedicated model server platform
2. Model serving platform descriptions
   Single model serving platform:
   Each model is deployed from its own model server. Only use this option for large language models that will be deployed using the Caikit runtime.

Multi-model: Multiple models are deployed from the same model server. Use this option when deploying small models that can share server resources.

3. Disabling model serving platforms info alerts
   Single is disabled:
   Disabling dedicated model serving prevents large language models from being deployed from projects on this cluster. You will only be able to deploy smaller models that can share a model server. Existing projects with deployed models will continue using their selected serving platform.

Multi is disabled: Disabling shared model serving prevents models from being deployed from shared model servers. Instead, they will use dedicated model servers. Existing projects with deployed models will continue using their selected serving platform.

Thanks for the suggestions, I like them. I have only some thoughts about them:

1. Only use this option for large language models that will be deployed using the Caikit runtime. this is likely going to change when we'll support multiple runtimes. So I would not mention Caikit runtime specifically
2. Only use this option for large language models ... why? Customers have the freedom to import their own Custom Serving Runtime and try deploying other non-LLM models. I'd rather re-write it into a "suggested usage" of the platform.
3. I feel like the alerts on disabling are too long and, especially, implicit if we have already provided a description of each platform. I think it would be cleaner just to say Existing projects with deployed models will continue using their selected serving platform

Wdyt?

[Xaenalt]

This is true, the platform does extend beyond LLMs if a user wants to install any of the other runtimes it supports. We are initially shipping with a very narrow set of things

[dgutride]

Migrated to https://issues.redhat.com/browse/RHOAIENG-543