openshift · adellape · Sep 27, 2024 · Sep 19, 2024
diff --git a/_attributes/common-attributes.adoc b/_attributes/common-attributes.adoc
@@ -12,6 +12,8 @@
 //Red Hat
 :red-hat: Red Hat
 :rhel: Red Hat Enterprise Linux
+:rhelai: Red Hat Enterprise Linux AI
+:rhoai: Red Hat OpenShift AI
 //OpenShift
 :ocp-product-title: OpenShift Container Platform
 :ocp-short-name: OpenShift

diff --git a/about/ols-about-openshift-lightspeed.adoc b/about/ols-about-openshift-lightspeed.adoc
@@ -16,8 +16,17 @@ include::modules/ols-large-language-model-requirements.adoc[leveloffset=+1]
 include::modules/ols-about-openai.adoc[leveloffset=+2]
 include::modules/ols-about-azure-openai.adoc[leveloffset=+2]
 include::modules/ols-about-watsonx.adoc[leveloffset=+2]
+include::modules/ols-about-rhoai.adoc[leveloffset=+2]
 include::modules/ols-about-data-use.adoc[leveloffset=+1]
 include::modules/ols-about-data-telemetry-transcript-and-feedback-collection.adoc[leveloffset=+1]
 include::modules/ols-about-remote-health-monitoring.adoc[leveloffset=+1]
 include::modules/ols-transcript-collection-overview.adoc[leveloffset=+2]
 include::modules/ols-feedback-collection-overview.adoc[leveloffset=+2]
+
+[id="additional-resources_{context}"]
+== Additional resources
+
+* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-the-credentials-secret-using-web-console_ols-configuring-openshift-lightspeed[Creating the credential secret using the web console]
+* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-the-credentials-secret-using-cli_ols-configuring-openshift-lightspeed[Creating the credential secret using the CLI]
+* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-lightspeed-custom-resource-file-using-web-console_ols-configuring-openshift-lightspeed[Creating the Lightspeed custom resource file using the web console]
+* xref:../configure/ols-configuring-openshift-lightspeed.adoc#ols-creating-lightspeed-custom-resource-file-using-cli_ols-configuring-openshift-lightspeed[Creating the Lightspeed custom resource file using the CLI]
diff --git a/modules/ols-about-rhelai.adoc b/modules/ols-about-rhelai.adoc
@@ -0,0 +1,9 @@
+// Module included in the following assemblies:
+
+// * about/ols-about-openshift-lightspeed.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="ols-about-rhelai_{context}"]
+= About {rhelai} 
+
+{rhelai} is OpenAI API-compatible, and is configured largely the same as the OpenAI provider. 
diff --git a/modules/ols-about-rhoai.adoc b/modules/ols-about-rhoai.adoc
@@ -0,0 +1,11 @@
+// Module included in the following assemblies:
+
+// * about/ols-about-openshift-lightspeed.adoc
+
+:_mod-docs-content-type: CONCEPT
+[id="ols-about-rhoai_{context}"]
+= About {rhoai} 
+
+{rhoai} is OpenAI API-compatible, and is configured largely the same as the OpenAI provider. 
+
+You need a Large Language Model (LLM) deployed on the single model-serving platform of {rhoai} using the Virtual Large Language Model (vLLM) runtime. If the model deployment is in a different {ocp-short-name} environment than the {ols-long} deployment, the model deployment must include a route to expose it outside the cluster. For more information, see link:https://docs.redhat.com/en/documentation/red_hat_openshift_ai_self-managed/2-latest/html/serving_models/serving-large-models_serving-large-models#about-the-single-model-serving-platform_serving-large-models[About the single-model serving platform].
diff --git a/modules/ols-creating-lightspeed-custom-resource-file-using-cli.adoc b/modules/ols-creating-lightspeed-custom-resource-file-using-cli.adoc
@@ -1,5 +1,6 @@
 // This module is used in the following assemblies:
-// configure/ols-configuring-openshift-lightspeed.adoc
+
+// * configure/ols-configuring-openshift-lightspeed.adoc
 
 :_mod-docs-content-type: PROCEDURE
 [id="ols-creating-lightspeed-custom-resource-file-using-cli_{context}"]
@@ -9,17 +10,16 @@ The Custom Resource (CR) file contains information that the Operator uses to dep
 
 .Prerequisites
 
-* You have access to the {ocp-short-name} CLI (oc) and are logged in as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped custom resource file.
+* You have access to the {ocp-short-name} CLI (oc) and are logged in as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped CR file.
 
 * You have installed the {ols-long} Operator.
 
 .Procedure
 
 . Create an `OLSConfig` file that contains the YAML content for the LLM provider you use:
 +
-.OpenAI custom resource file
-+
-[source,yaml, subs="attributes,verbatim"]
+.OpenAI CR file
+[source,yaml,subs="attributes,verbatim"]
 ----
 apiVersion: ols.openshift.io/v1alpha1
 kind: OLSConfig
@@ -32,22 +32,40 @@ spec:
         type: openai
         credentialsSecretRef:
           name: credentials
-        url: "https://api.openai.com/v1"
+        url: https://api.openai.com/v1
         models:
           - name: gpt-3.5-turbo
   ols:
     defaultModel: gpt-3.5-turbo
     defaultProvider: myOpenai
 ----
 +
-[NOTE]
-====
-For OpenShift AI vLLM use the same configuration as OpenAI but update the URL to point to your Virtual Large Language Model (vLLM) endpoint. If OpenShift Lightspeed operates in the same cluster as the vLLM model serving instance, you can point to the internal OpenShift service hostname instead of exposing vLLM with a route.
-====
-+
-.{azure-openai} custom resource file
+.{rhoai} CR file
+[source,yaml,subs="attributes,verbatim"]
+----
+apiVersion: ols.openshift.io/v1alpha1
+kind: OLSConfig
+metadata:
+  name: cluster
+  namespace: openshift-lightspeed
+spec:
+  llm:
+    providers:
+    - credentialsSecretRef:
+        name: openai-api-keys
+      models:
+      - name: granite-8b-code-instruct-128k
+      name: red_hat_openshift_ai
+      type: red_hat_openshift_ai_vllm
+      url: <url> <1>
+  ols:
+    defaultProvider: red_hat_openshift_ai
+    defaultModel: granite-8b-code-instruct-128k
+----
+<1> The URL endpoint must end with `v1` to be valid. For example, `\https://granite-8b-code-instruct.my-domain.com:443/v1`. 
 +
-[source,yaml, subs="attributes,verbatim"]
+.{azure-openai} CR file
+[source,yaml,subs="attributes,verbatim"]
 ----
 apiVersion: ols.openshift.io/v1alpha1
 kind: OLSConfig
@@ -67,12 +85,10 @@ spec:
   ols:
     defaultModel: gpt-35-turbo-16k
     defaultProvider: myAzure
-    logLevel: DEBUG
 ----
 +
-.{watsonx} custom resource file
-+
-[source,yaml, subs="attributes,verbatim"]
+.{watsonx} CR file
+[source,yaml,subs="attributes,verbatim"]
 ----
 apiVersion: ols.openshift.io/v1alpha1
 kind: OLSConfig
@@ -92,7 +108,6 @@ spec:
   ols:
     defaultModel: ibm/granite-13b-chat-v2
     defaultProvider: myWatsonx
-    logLevel: DEBUG
 ----
 
 . Run the following command:

diff --git a/modules/ols-creating-lightspeed-custom-resource-file-using-web-console.adoc b/modules/ols-creating-lightspeed-custom-resource-file-using-web-console.adoc
@@ -1,5 +1,6 @@
 // This module is used in the following assemblies:
-// configure/ols-configuring-openshift-lightspeed.adoc
+
+// * configure/ols-configuring-openshift-lightspeed.adoc
 
 :_mod-docs-content-type: PROCEDURE
 [id="ols-creating-lightspeed-custom-resource-file-using-web-console_{context}"]
@@ -9,19 +10,18 @@ The Custom Resource (CR) file contains information that the Operator uses to dep
 
 .Prerequisites
 
-* You are logged in to the {ocp-product-title} web console as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped custom resource file.
+* You are logged in to the {ocp-product-title} web console as a user with the `cluster-admin` role. Alternatively, you are logged in to a user account that has permission to create a cluster-scoped CR file.
 
 * You have installed the {ols-long} Operator.
 
 .Procedure 
 
-. Click the plus button in the upper-right corner of the {ocp-short-name} web console.
+. Click *Add* in the upper-right corner of the {ocp-short-name} web console.
 
 . Paste the YAML content for the LLM provider you use into the text area of the web console:
 +
-.OpenAI custom resource file
-+
-[source,yaml, subs="attributes,verbatim"]
+.OpenAI CR file
+[source,yaml,subs="attributes,verbatim"]
 ----
 apiVersion: ols.openshift.io/v1alpha1
 kind: OLSConfig
@@ -34,17 +34,40 @@ spec:
         type: openai
         credentialsSecretRef:
           name: credentials
-        url: "https://api.openai.com/v1"
+        url: https://api.openai.com/v1
         models:
           - name: gpt-3.5-turbo
   ols:
     defaultModel: gpt-3.5-turbo
     defaultProvider: myOpenai
 ----
 +
-.{azure-openai} custom resource file
+.{rhoai} CR file
+[source,yaml,subs="attributes,verbatim"]
+----
+apiVersion: ols.openshift.io/v1alpha1
+kind: OLSConfig
+metadata:
+  name: cluster
+  namespace: openshift-lightspeed
+spec:
+  llm:
+    providers:
+    - credentialsSecretRef:
+        name: openai-api-keys
+      models:
+      - name: granite-8b-code-instruct-128k
+      name: red_hat_openshift_ai
+      type: red_hat_openshift_ai_vllm
+      url: <url> <1>
+  ols:
+    defaultProvider: red_hat_openshift_ai
+    defaultModel: granite-8b-code-instruct-128k
+----
+<1> The URL endpoint must end with `v1` to be valid. For example, `\https://granite-8b-code-instruct.my-domain.com:443/v1`. 
 +
-[source,yaml, subs="attributes,verbatim"]
+.{azure-openai} CR file
+[source,yaml,subs="attributes,verbatim"]
 ----
 apiVersion: ols.openshift.io/v1alpha1
 kind: OLSConfig
@@ -66,9 +89,8 @@ spec:
     defaultProvider: myAzure
 ----
 +
-.{watsonx} custom resource file
-+
-[source,yaml, subs="attributes,verbatim"]
+.{watsonx} CR file
+[source,yaml,subs="attributes,verbatim"]
 ----
 apiVersion: ols.openshift.io/v1alpha1
 kind: OLSConfig

diff --git a/modules/ols-large-language-model-requirements.adoc b/modules/ols-large-language-model-requirements.adoc
@@ -1,18 +1,23 @@
+// This module is used in the following assemblies:
+
+// * about/ols-about-openshift-lightspeed.adoc
+
 :_mod-docs-content-type: CONCEPT
 [id="ols-large-language-model-requirements"]
 = Large Language Model (LLM) requirements 
 :context: ols-large-language-model-requirements
 
-As part of the {ols-release}  release, {ols-long} relies on Software as a Service (SaaS) LLM providers. You will need to have either a trial or paid subscription that allows for API access to completions and inferences with one of the following providers:
+As part of the {ols-release} release, {ols-long} can rely on the following Software as a Service (SaaS) Large Language Model (LLM) providers: 
 
 * OpenAI
 
-* Azure OpenAI
+* {azure-openai}
 
-* WatsonX
+* {watsonx}
 
 [NOTE]
 ====
 Many self-hosted or self-managed model servers claim API compatibility with OpenAI. It is possible to configure the OpenShift Lightspeed OpenAI provider to point to an API-compatible model server. If the model server is truly API-compatible, especially with respect to authentication, then it may work. These configurations have not been tested by Red Hat, and issues related to their use are outside the scope of {ols-release} support.
 ====
 
+For {ols-long} configurations with {rhoai}, you must host your own LLM provider.