diff --git a/_attributes/common-attributes.adoc b/_attributes/common-attributes.adoc index 7f7b92a845c9..dac4d3a61bf1 100644 --- a/_attributes/common-attributes.adoc +++ b/_attributes/common-attributes.adoc @@ -12,6 +12,8 @@ //Red Hat :red-hat: Red Hat :rhel: Red Hat Enterprise Linux +:rhelai: Red Hat Enterprise Linux AI +:rhoai: Red Hat OpenShift AI //OpenShift :ocp-product-title: OpenShift Container Platform :ocp-short-name: OpenShift diff --git a/about/ols-about-openshift-lightspeed.adoc b/about/ols-about-openshift-lightspeed.adoc index 7394b6d28d36..f3d16e183a4a 100644 --- a/about/ols-about-openshift-lightspeed.adoc +++ b/about/ols-about-openshift-lightspeed.adoc @@ -16,8 +16,17 @@ include::modules/ols-large-language-model-requirements.adoc[leveloffset=+1] include::modules/ols-about-openai.adoc[leveloffset=+2] include::modules/ols-about-azure-openai.adoc[leveloffset=+2] include::modules/ols-about-watsonx.adoc[leveloffset=+2] +include::modules/ols-about-rhoai.adoc[leveloffset=+2] include::modules/ols-about-data-use.adoc[leveloffset=+1] include::modules/ols-about-data-telemetry-transcript-and-feedback-collection.adoc[leveloffset=+1] include::modules/ols-about-remote-health-monitoring.adoc[leveloffset=+1] include::modules/ols-transcript-collection-overview.adoc[leveloffset=+2] include::modules/ols-feedback-collection-overview.adoc[leveloffset=+2] + +[id="additional-resources_{context}"] +== Additional resources + +* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-the-credentials-secret-using-web-console_ols-configuring-openshift-lightspeed[Creating the credential secret using the web console] +* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-the-credentials-secret-using-cli_ols-configuring-openshift-lightspeed[Creating the credential secret using the CLI] +* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-lightspeed-custom-resource-file-using-web-console_ols-configuring-openshift-lightspeed[Creating the Lightspeed custom resource file using the web console] +* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-lightspeed-custom-resource-file-using-cli_ols-configuring-openshift-lightspeed[Creating the Lightspeed custom resource file using the CLI] \ No newline at end of file diff --git a/modules/ols-about-rhelai.adoc b/modules/ols-about-rhelai.adoc new file mode 100644 index 000000000000..bad3ac510f62 --- /dev/null +++ b/modules/ols-about-rhelai.adoc @@ -0,0 +1,9 @@ +// Module included in the following assemblies: + +// * about/ols-about-openshift-lightspeed.adoc + +:_mod-docs-content-type: CONCEPT +[id="ols-about-rhelai_{context}"] += About {rhelai} + +{rhelai} is OpenAI API-compatible, and is configured largely the same as the OpenAI provider. diff --git a/modules/ols-about-rhoai.adoc b/modules/ols-about-rhoai.adoc new file mode 100644 index 000000000000..11383284dbfa --- /dev/null +++ b/modules/ols-about-rhoai.adoc @@ -0,0 +1,11 @@ +// Module included in the following assemblies: + +// * about/ols-about-openshift-lightspeed.adoc + +:_mod-docs-content-type: CONCEPT +[id="ols-about-rhoai_{context}"] += About {rhoai} + +{rhoai} is OpenAI API-compatible, and is configured largely the same as the OpenAI provider. + +You need a Large Language Model (LLM) deployed on the single model-serving platform of {rhoai} using the Virtual Large Language Model (vLLM) runtime. If the model deployment is in a different {ocp-short-name} environment than the {ols-long} deployment, the model deployment must include a route to expose it outside the cluster. For more information, see link:https://docs.redhat.com/en/documentation/red_hat_openshift_ai_self-managed/2-latest/html/serving_models/serving-large-models_serving-large-models#about-the-single-model-serving-platform_serving-large-models[About the single-model serving platform]. diff --git a/modules/ols-creating-lightspeed-custom-resource-file-using-cli.adoc b/modules/ols-creating-lightspeed-custom-resource-file-using-cli.adoc index 96aa7dfa7269..f89610eb0237 100644 --- a/modules/ols-creating-lightspeed-custom-resource-file-using-cli.adoc +++ b/modules/ols-creating-lightspeed-custom-resource-file-using-cli.adoc @@ -1,5 +1,6 @@ // This module is used in the following assemblies: -// configure/ols-configuring-openshift-lightspeed.adoc + +// * configure/ols-configuring-openshift-lightspeed.adoc :_mod-docs-content-type: PROCEDURE [id="ols-creating-lightspeed-custom-resource-file-using-cli_{context}"] @@ -9,7 +10,7 @@ The Custom Resource (CR) file contains information that the Operator uses to dep .Prerequisites -* You have access to the {ocp-short-name} CLI (oc) and are logged in as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped custom resource file. +* You have access to the {ocp-short-name} CLI (oc) and are logged in as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped CR file. * You have installed the {ols-long} Operator. @@ -17,9 +18,8 @@ The Custom Resource (CR) file contains information that the Operator uses to dep . Create an `OLSConfig` file that contains the YAML content for the LLM provider you use: + -.OpenAI custom resource file -+ -[source,yaml, subs="attributes,verbatim"] +.OpenAI CR file +[source,yaml,subs="attributes,verbatim"] ---- apiVersion: ols.openshift.io/v1alpha1 kind: OLSConfig @@ -32,7 +32,7 @@ spec: type: openai credentialsSecretRef: name: credentials - url: "https://api.openai.com/v1" + url: https://api.openai.com/v1 models: - name: gpt-3.5-turbo ols: @@ -40,14 +40,32 @@ spec: defaultProvider: myOpenai ---- + -[NOTE] -==== -For OpenShift AI vLLM use the same configuration as OpenAI but update the URL to point to your Virtual Large Language Model (vLLM) endpoint. If OpenShift Lightspeed operates in the same cluster as the vLLM model serving instance, you can point to the internal OpenShift service hostname instead of exposing vLLM with a route. -==== -+ -.{azure-openai} custom resource file +.{rhoai} CR file +[source,yaml,subs="attributes,verbatim"] +---- +apiVersion: ols.openshift.io/v1alpha1 +kind: OLSConfig +metadata: + name: cluster + namespace: openshift-lightspeed +spec: + llm: + providers: + - credentialsSecretRef: + name: openai-api-keys + models: + - name: granite-8b-code-instruct-128k + name: red_hat_openshift_ai + type: red_hat_openshift_ai_vllm + url: <1> + ols: + defaultProvider: red_hat_openshift_ai + defaultModel: granite-8b-code-instruct-128k +---- +<1> The URL endpoint must end with `v1` to be valid. For example, `\https://granite-8b-code-instruct.my-domain.com:443/v1`. + -[source,yaml, subs="attributes,verbatim"] +.{azure-openai} CR file +[source,yaml,subs="attributes,verbatim"] ---- apiVersion: ols.openshift.io/v1alpha1 kind: OLSConfig @@ -67,12 +85,10 @@ spec: ols: defaultModel: gpt-35-turbo-16k defaultProvider: myAzure - logLevel: DEBUG ---- + -.{watsonx} custom resource file -+ -[source,yaml, subs="attributes,verbatim"] +.{watsonx} CR file +[source,yaml,subs="attributes,verbatim"] ---- apiVersion: ols.openshift.io/v1alpha1 kind: OLSConfig @@ -92,7 +108,6 @@ spec: ols: defaultModel: ibm/granite-13b-chat-v2 defaultProvider: myWatsonx - logLevel: DEBUG ---- . Run the following command: diff --git a/modules/ols-creating-lightspeed-custom-resource-file-using-web-console.adoc b/modules/ols-creating-lightspeed-custom-resource-file-using-web-console.adoc index 5abc5f0eedf6..574d7bb70fad 100644 --- a/modules/ols-creating-lightspeed-custom-resource-file-using-web-console.adoc +++ b/modules/ols-creating-lightspeed-custom-resource-file-using-web-console.adoc @@ -1,5 +1,6 @@ // This module is used in the following assemblies: -// configure/ols-configuring-openshift-lightspeed.adoc + +// * configure/ols-configuring-openshift-lightspeed.adoc :_mod-docs-content-type: PROCEDURE [id="ols-creating-lightspeed-custom-resource-file-using-web-console_{context}"] @@ -9,19 +10,18 @@ The Custom Resource (CR) file contains information that the Operator uses to dep .Prerequisites -* You are logged in to the {ocp-product-title} web console as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped custom resource file. +* You are logged in to the {ocp-product-title} web console as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped CR file. * You have installed the {ols-long} Operator. .Procedure -. Click the plus button in the upper-right corner of the {ocp-short-name} web console. +. Click *Add* in the upper-right corner of the {ocp-short-name} web console. . Paste the YAML content for the LLM provider you use into the text area of the web console: + -.OpenAI custom resource file -+ -[source,yaml, subs="attributes,verbatim"] +.OpenAI CR file +[source,yaml,subs="attributes,verbatim"] ---- apiVersion: ols.openshift.io/v1alpha1 kind: OLSConfig @@ -34,7 +34,7 @@ spec: type: openai credentialsSecretRef: name: credentials - url: "https://api.openai.com/v1" + url: https://api.openai.com/v1 models: - name: gpt-3.5-turbo ols: @@ -42,9 +42,32 @@ spec: defaultProvider: myOpenai ---- + -.{azure-openai} custom resource file +.{rhoai} CR file +[source,yaml,subs="attributes,verbatim"] +---- +apiVersion: ols.openshift.io/v1alpha1 +kind: OLSConfig +metadata: + name: cluster + namespace: openshift-lightspeed +spec: + llm: + providers: + - credentialsSecretRef: + name: openai-api-keys + models: + - name: granite-8b-code-instruct-128k + name: red_hat_openshift_ai + type: red_hat_openshift_ai_vllm + url: <1> + ols: + defaultProvider: red_hat_openshift_ai + defaultModel: granite-8b-code-instruct-128k +---- +<1> The URL endpoint must end with `v1` to be valid. For example, `\https://granite-8b-code-instruct.my-domain.com:443/v1`. + -[source,yaml, subs="attributes,verbatim"] +.{azure-openai} CR file +[source,yaml,subs="attributes,verbatim"] ---- apiVersion: ols.openshift.io/v1alpha1 kind: OLSConfig @@ -66,9 +89,8 @@ spec: defaultProvider: myAzure ---- + -.{watsonx} custom resource file -+ -[source,yaml, subs="attributes,verbatim"] +.{watsonx} CR file +[source,yaml,subs="attributes,verbatim"] ---- apiVersion: ols.openshift.io/v1alpha1 kind: OLSConfig diff --git a/modules/ols-large-language-model-requirements.adoc b/modules/ols-large-language-model-requirements.adoc index bba0c32f9076..6695e2cb7c2c 100644 --- a/modules/ols-large-language-model-requirements.adoc +++ b/modules/ols-large-language-model-requirements.adoc @@ -1,18 +1,23 @@ +// This module is used in the following assemblies: + +// * about/ols-about-openshift-lightspeed.adoc + :_mod-docs-content-type: CONCEPT [id="ols-large-language-model-requirements"] = Large Language Model (LLM) requirements :context: ols-large-language-model-requirements -As part of the {ols-release} release, {ols-long} relies on Software as a Service (SaaS) LLM providers. You will need to have either a trial or paid subscription that allows for API access to completions and inferences with one of the following providers: +As part of the {ols-release} release, {ols-long} can rely on the following Software as a Service (SaaS) Large Language Model (LLM) providers: * OpenAI -* Azure OpenAI +* {azure-openai} -* WatsonX +* {watsonx} [NOTE] ==== Many self-hosted or self-managed model servers claim API compatibility with OpenAI. It is possible to configure the OpenShift Lightspeed OpenAI provider to point to an API-compatible model server. If the model server is truly API-compatible, especially with respect to authentication, then it may work. These configurations have not been tested by Red Hat, and issues related to their use are outside the scope of {ols-release} support. ==== +For {ols-long} configurations with {rhoai}, you must host your own LLM provider.