[Docs] Sensor Agent Doc (#4195)

* sensor agent doc Signed-off-by: Future Outlier <eric901201@gmai.com> * sensor agent index rst Signed-off-by: Future Outlier <eric901201@gmai.com> --------- Signed-off-by: Future Outlier <eric901201@gmai.com> Co-authored-by: Future Outlier <eric901201@gmai.com> Co-authored-by: Kevin Su <pingsutw@apache.org>
flyteorg · Nov 11, 2023 · 0fc2ab0 · 0fc2ab0
1 parent 31748fe
commit 0fc2ab0
Show file tree

Hide file tree

Showing 3 changed files with 189 additions and 19 deletions.
diff --git a/rsts/deployment/agents/bigquery.rst b/rsts/deployment/agents/bigquery.rst
@@ -20,30 +20,39 @@ Specify agent configuration
 
 .. tabs::
 
-  .. group-tab:: Flyte binary
+    .. group-tab:: Flyte binary
 
-    Edit the relevant YAML file to specify the agent.
+      Edit the relevant YAML file to specify the agent.
 
-    .. code-block:: yaml
-      :emphasize-lines: 7,11
+      .. code-block:: bash
 
-      tasks:
-        task-plugins:
-          enabled-plugins:
-            - container
-            - sidecar
-            - k8s-array
-            - bigquery
-          default-for-task-types:
-            - container: container
-            - container_array: k8s-array
-            - bigquery_query_job_task: agent-service
+        kubectl edit configmap flyte-sandbox-config -n flyte
 
-  .. group-tab:: Flyte core
+      .. code-block:: yaml
+        :emphasize-lines: 7,11,16
+  
+        tasks:
+          task-plugins:
+            enabled-plugins:
+              - container
+              - sidecar
+              - k8s-array
+              - agent-service
+            default-for-task-types:
+              - container: container
+              - container_array: k8s-array
+              - bigquery_query_job_task: agent-service
+        
+        plugins:
+          agent-service:
+            supportedTaskTypes:
+            - bigquery_query_job_task
+
+    .. group-tab:: Flyte core
 
-    Create a file named ``values-override.yaml`` and add the following configuration to it.
+      Create a file named ``values-override.yaml`` and add the following configuration to it.
 
-    .. code-block:: yaml
+      .. code-block:: yaml
 
         configmap:
           enabled_plugins:
@@ -56,12 +65,16 @@ Specify agent configuration
                   - container
                   - sidecar
                   - k8s-array
-                  - bigquery
+                  - agent-service
                 default-for-task-types:
                   container: container
                   sidecar: sidecar
                   container_array: k8s-array
                   bigquery_query_job_task: agent-service
+            plugins:
+              agent-service:
+                supportedTaskTypes:
+                - bigquery_query_job_task
 
 Ensure that the propeller has the correct service account for BigQuery.
 

diff --git a/rsts/deployment/agents/index.rst b/rsts/deployment/agents/index.rst
@@ -27,6 +27,15 @@ Discover the process of setting up Agents for Flyte.
     ^^^^^^^^^^^^
     Guide to setting up the MMCloud agent.
 
+    ---
+
+    .. link-button:: deployment-agent-setup-sensor
+       :type: ref
+       :text: Sensor Agent
+       :classes: btn-block stretched-link
+    ^^^^^^^^^^^^
+    Guide to setting up the Sensor agent.
+
 
 .. toctree::
     :maxdepth: 1
@@ -35,3 +44,4 @@ Discover the process of setting up Agents for Flyte.
 
     bigquery
     mmcloud
+    sensor
diff --git a/rsts/deployment/agents/sensor.rst b/rsts/deployment/agents/sensor.rst
@@ -0,0 +1,147 @@
+.. _deployment-agent-setup-sensor:
+
+Sensor Agent
+=================
+
+Sensor enables users to continuously check for a file or a condition to be met periodically.
+
+When the condition is met, the sensor will complete.
+
+This guide provides an overview of how to set up Sensor in your Flyte deployment.
+
+Spin up a cluster
+-----------------
+
+.. tabs::
+
+  .. group-tab:: Flyte binary
+
+    You can spin up a demo cluster using the following command:
+
+    .. code-block:: bash
+
+      flytectl demo start
+
+    Or install Flyte using the :ref:`flyte-binary helm chart <deployment-deployment-cloud-simple>`.
+
+  .. group-tab:: Flyte core
+
+    If you've installed Flyte using the
+    `flyte-core helm chart <https://github.com/flyteorg/flyte/tree/master/charts/flyte-core>`__, please ensure:
+
+    * You have the correct kubeconfig and have selected the correct Kubernetes context.
+    * Confirm that you have the correct Flytectl configuration at ``~/.flyte/config.yaml``.
+
+.. note::
+
+  Add the Flyte chart repo to Helm if you're installing via the Helm charts.
+
+  .. code-block:: bash
+
+    helm repo add flyteorg https://flyteorg.github.io/flyte
+
+Specify agent configuration
+----------------------------
+
+Enable the Sensor agent by adding the following config to the relevant YAML file(s):
+
+.. tabs::
+
+    .. group-tab:: Flyte binary
+
+      Edit the relevant YAML file to specify the agent.
+
+      .. code-block:: bash
+
+        kubectl edit configmap flyte-sandbox-config -n flyte
+
+      .. code-block:: yaml
+        :emphasize-lines: 7,11,16
+  
+        tasks:
+          task-plugins:
+            enabled-plugins:
+              - container
+              - sidecar
+              - k8s-array
+              - agent-service
+            default-for-task-types:
+              - container: container
+              - container_array: k8s-array
+              - sensor: agent-service
+        
+        plugins:
+          agent-service:
+            supportedTaskTypes:
+            - sensor
+
+    .. group-tab:: Flyte core
+
+      Create a file named ``values-override.yaml`` and add the following configuration to it.
+
+      .. code-block:: yaml
+
+        configmap:
+          enabled_plugins:
+            # -- Tasks specific configuration [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#GetConfig)
+            tasks:
+              # -- Plugins configuration, [structure](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/nodes/task/config#TaskPluginConfig)
+              task-plugins:
+                # -- [Enabled Plugins](https://pkg.go.dev/github.com/flyteorg/flyteplugins/go/tasks/config#Config). Enable sagemaker*, athena if you install the backend
+                enabled-plugins:
+                  - container
+                  - sidecar
+                  - k8s-array
+                  - agent-service
+                default-for-task-types:
+                  container: container
+                  sidecar: sidecar
+                  container_array: k8s-array
+                  sensor: agent-service
+            plugins:
+              agent-service:
+                supportedTaskTypes:
+                - sensor
+
+
+Upgrade the deployment
+----------------------
+
+.. tabs::
+
+  .. group-tab:: Flyte binary
+
+    .. tabs::
+
+      .. group-tab:: Demo cluster
+
+        .. code-block:: bash
+
+          kubectl rollout restart deployment flyte-sandbox -n flyte
+
+      .. group-tab:: Helm chart
+
+        .. code-block:: bash
+
+          helm upgrade <RELEASE_NAME> flyteorg/flyte-binary -n <YOUR_NAMESPACE> --values <YOUR_YAML_FILE>
+
+        Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte-backend``),
+        ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``),
+        and ``<YOUR_YAML_FILE>`` with the name of your YAML file.
+
+  .. group-tab:: Flyte core
+
+    .. code-block::
+
+      helm upgrade <RELEASE_NAME> flyte/flyte-core -n <YOUR_NAMESPACE> --values values-override.yaml
+
+    Replace ``<RELEASE_NAME>`` with the name of your release (e.g., ``flyte``)
+    and ``<YOUR_NAMESPACE>`` with the name of your namespace (e.g., ``flyte``).
+
+Wait for the upgrade to complete.
+
+You can check the status of the deployment pods by running the following command:
+
+.. code-block::
+
+  kubectl get pods -n flyte