feat(chart): Configuration extra scripts mount to container (#2105)

* feat(chart): Configuration extra scripts mount to container

Signed-off-by: Viet Nguyen Duc <nguyenducviet4496@gmail.com>

* Update Chart table of contents
[skip ci]

Signed-off-by: Viet Nguyen Duc <nguyenducviet4496@gmail.com>

* Video image is missing in CI test

Signed-off-by: Viet Nguyen Duc <nguyenducviet4496@gmail.com>

---------

Signed-off-by: Viet Nguyen Duc <nguyenducviet4496@gmail.com>

charts/selenium-grid/README.md

@@ -18,10 +18,11 @@ This chart enables the creation of a Selenium Grid Server in Kubernetes.
   * [Ingress Configuration](#ingress-configuration)
   * [Configuration](#configuration)
     * [Configuration global](#configuration-global)
-      * [Configuration `global.K8S_PUBLIC_IP`](#configuration-globalk8s_public_ip)
+      * [Configuration `global.K8S_PUBLIC_IP`](#configuration-globalk8spublicip)
     * [Configuration of Nodes](#configuration-of-nodes)
       * [Container ports and Service ports](#container-ports-and-service-ports)
       * [Probes](#probes)
+    * [Configuration extra scripts mount to container](#configuration-extra-scripts-mount-to-container)
     * [Configuration of video recorder and video uploader](#configuration-of-video-recorder-and-video-uploader)
       * [Video recorder](#video-recorder)
       * [Video uploader](#video-uploader)
@@ -378,6 +379,52 @@ edgeNode:
     periodSeconds: 5
 ```
 
+### Configuration extra scripts mount to container
+
+This is supported for containers of browser node, video recorder and video uploader. By default, in these containers, there are scripts, config files implemented. In case you want to customize or replace them with your own implementation. Instead of forking the chart, use volume mount. Now, from your external files, you can insert them into ConfigMap via Helm CLI `--set-file` or compose them in your own YAML values file and pass to Helm CLI `--values` when deploying chart. Any files name that you defined will be picked up into ConfigMap and mounted to the container.
+
+```yaml
+nodeConfigMap:
+  extraScriptsDirectory: "/opt/selenium"
+  extraScripts:
+    nodePreStop.sh: |
+      #!/bin/bash
+      echo "Your custom script"
+
+recorderConfigMap:
+  extraScriptsDirectory: "/opt/selenium"
+  extraScripts:
+    video.sh: |
+        #!/bin/bash
+        echo "Your custom script"    
+    videoPreStop.sh: |
+        #!/bin/bash
+        echo "My new script"
+
+uploaderConfigMap:
+  extraScriptsDirectory: "/opt/bin"
+  extraScripts:
+    entry_point.sh: |
+        #!/bin/bash
+        echo "Your custom entry point"
+  secretFiles:
+    config.conf: |
+        [myremote]
+        type = s3
+```
+
+Via Helm CLI, you can pass your own files to particular config key. Note that, the file name contains dot `.` for file extension, it will impact to the key name convention in Helm CLI. In this case, be careful to escape the dot `.` in the file name. For example a command in Unix:
+
+```bash
+helm upgrade -i test \
+    --set-file 'nodeConfigMap.extraScripts.nodePreStop\.sh=/path/to/myScript.sh' \
+    --set-file 'recorderConfigMap.extraScripts.video\.sh=/path/to/myCustom.sh' \
+    selenium-grid
+```
+
+Files in `.extraScripts` will be mounted to the container with the same name within directory is defined in `.extraScriptsDirectory`. For example, in the above config, `nodePreStop.sh` will be mounted to `/opt/selenium/nodePreStop.sh` in the node container.
+
+
 ### Configuration of video recorder and video uploader
 
 #### Video recorder
@@ -389,6 +436,20 @@ videoRecorder:
   enabled: true
 ```
 
+At chart deployment level, that config will enable video container always. In addition, you can disable video recording process via session capability `se:recordVideo`. For example in Python binding:
+
+```python
+from selenium.webdriver.chrome.options import Options as ChromeOptions
+from selenium import webdriver
+
+options = ChromeOptions()
+options.set_capability('se:recordVideo', False)
+driver = webdriver.Remote(options=options, command_executor="http://localhost:4444")
+)
+```
+
+In Node will perform query GraphQL and extract the value of `se:recordVideo` in capabilities before deciding to start video recording process or not. By default, the script is loaded from file [configs/recorder/graphQLRecordVideo.sh](configs/recorder/graphQLRecordVideo.sh) to the ConfigMap. You can customize by reading on section [Configuration extra scripts mount to container](#configuration-extra-scripts-mount-to-container).
+
 #### Video uploader
 
 The uploader is a sidecar that is deployed with the browser nodes. It is responsible for uploading the video to a remote location. The uploader is disabled by default. To enable it, you need to set the following values:
@@ -401,17 +462,16 @@ videoRecorder:
 
 By default, the uploader uses [RCLONE](https://rclone.org/) to upload the video to a remote location. RCLONE requires a configuration file to define different remote locations. Refer to [RCLONE docs](https://rclone.org/docs/#config-file) for more details. Config file might contain sensitive information such as access key, secret key, etc. hence it is stored in Secret.
 
-The uploader requires `destinationPrefix` to be set. It is used to instruct the uploader where to upload the video. The format of destinationPrefix is `remote-name://bucket-name/path`. The `remote-name` is configured in RCLONE. The bucket-name is the name of the bucket in the remote location. The path is the path to the folder in the bucket.
+The uploader requires `destinationPrefix` to be set. It is used to instruct the uploader where to upload the video. The format of destinationPrefix is `remote-name://bucket-name/path`. The `remote-name` is configured in RCLONE. The `bucket-name` is the name of the bucket in the remote location. The `path` is the path to the folder in the bucket.
 
-By default, the config file is loaded from file [configs/uploader/rclone/rclone.conf](configs/uploader/rclone/rclone.conf) to the Secret. You can override the config file via `--set-file videoRecorder.uploader.config=/path/to/config` or set via YAML values.
+By default, the config file is loaded from file [configs/uploader/rclone/config.conf](configs/uploader/rclone/config.conf) to the Secret. You can override the config file via `--set-file uploaderConfigMap.secretFiles.config.conf=/path/to/your_config.conf` or set via YAML values.
 
 For example, to configure an S3 remote hosted on AWS with named `mys3` and the bucket name is `mybucket`, you can set the following values:
 
 ```bash
-videoRecorder:
-  uploader:
-    destinationPrefix: "mys3://mybucket"
-    config: |
+uploaderConfigMap:
+  secretFiles:
+    config.conf: |
         [mys3]
         type = s3
         provider = AWS
@@ -421,10 +481,14 @@ videoRecorder:
         acl = private
         access_key_id = xxx
         secret_access_key = xxx
+
+videoRecorder:
+  uploader:
+    destinationPrefix: "mys3://mybucket/subFolder"
 ```
 
 You can prepare a config file with multiple remotes are defined. Ensure that `[remoteName]` is unique for each remote.
-You also can replace your config to default file `configs/uploader/rclone/rclone.conf` in chart.
+You also can replace your config to default file `configs/uploader/rclone/config.conf` in chart.
 
 Instead of using config file, another way that RCLONE also supports to pass the information via environment variables. ENV variable with format: `RCLONE_CONFIG_ + name of remote + _ + name of config file option` (make it all uppercase). In this case the remote name it can only contain letters, digits, or the _ (underscore) character. All those ENV variables can be set via `videoRecorder.uploader.secrets`, it will be stored in Secret.
 
@@ -445,16 +509,16 @@ videoRecorder:
       RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY: "xxx"
 ```
 
-Those 2 ways are equivalent. You can choose one of them or combine them together. When both config file and ENV vars are set, value in `rclone.conf` will take precedence.
+Those 2 ways are equivalent. You can choose one of them or combine them together. When both config file and ENV vars are set, value in `config.conf` will take precedence.
 
-Beside the configuration, the script for entry point of uploader container also needed. By default, it is loaded from file [configs/uploader/rclone/entry_point.sh](configs/uploader/rclone/entry_point.sh) to the ConfigMap. You can override the script via `--set-file videoRecorder.uploader.entryPoint=/path/to/script` or set via YAML values. For example:
+Beside the configuration, the script for entry point of uploader container also needed. By default, it is loaded from file [configs/uploader/rclone/entry_point.sh](configs/uploader/rclone/entry_point.sh) to the ConfigMap. You can override the script via `--set-file uploaderConfigMap.extraScripts.entry_point.sh=/path/to/your_script.sh` or set via YAML values. For example:
 
 ```yaml
-videoRecorder:
-  uploader:
-    entryPoint: |
+uploaderConfigMap:
+  extraScripts:
+    entry_point.sh: |
         #!/bin/bash
-        echo "Your custom script"
+        echo "Your custom entry point"
 ```
 
 You also can replace your script to default file `configs/uploader/rclone/entry_point.sh` in chart.
@@ -707,7 +771,7 @@ This table contains the configuration parameters of the chart and their default
 | `videoRecorder.uploader.enabled`              | `false`                                     | Enable the uploader for videos                                                                                             |
 | `videoRecorder.uploader.destinationPrefix`    | ``                                          | Destination for uploading video file. It is following `rclone` config                                                      |
 | `videoRecorder.uploader.name`                 | `rclone`                                    | Name of the uploader to use. Supported default `rclone`                                                                    |
-| `videoRecorder.uploader.configFileName`       | `rclone.conf`                               | Config file name for `rclone` in uploader container                                                                        |
+| `videoRecorder.uploader.configFileName`       | `config.conf`                               | Config file name for `rclone` in uploader container                                                                        |
 | `videoRecorder.uploader.entryPointFileName`   | `entry_point.sh`                            | Script file name for uploader container entry point                                                                        |
 | `videoRecorder.uploader.config`               | ``                                          | Set value to uploader config file via YAML or `--set-file`                                                                 |
 | `videoRecorder.uploader.entryPoint`           | ``                                          | Set value to uploader entry point via YAML or `--set-file`                                                                 |

charts/selenium-grid/configs/uploader/rclone/entry_point.sh

@@ -2,7 +2,7 @@
 
 SE_VIDEO_FOLDER=${SE_VIDEO_FOLDER:-"/videos"}
 UPLOAD_CONFIG_DIRECTORY=${UPLOAD_CONFIG_DIRECTORY:-"/opt/bin"}
-UPLOAD_CONFIG_FILE_NAME=${UPLOAD_CONFIG_FILE_NAME:-"rclone.conf"}
+UPLOAD_CONFIG_FILE_NAME=${UPLOAD_CONFIG_FILE_NAME:-"config.conf"}
 UPLOAD_COMMAND=${UPLOAD_COMMAND:-"copy"}
 
 function consume_force_exit() {

charts/selenium-grid/templates/node-configmap.yaml

@@ -16,10 +16,14 @@ data:
   SE_NODE_GRID_URL: '{{ include "seleniumGrid.url" .}}'
   SE_NODE_GRID_GRAPHQL_URL: '{{ include "seleniumGrid.graphqlURL" .}}'
 {{- range $fileName, $value := .Values.nodeConfigMap.extraScripts }}
-  {{- if not (empty $value) }}
+{{- if not (empty $value) }}
 {{ $fileName | indent 2 -}}: |
 {{ $value | indent 4 }}
-  {{- else }}
-{{ ($.Files.Glob (printf "configs/node/%s" $fileName)).AsConfig | indent 2 }}
-  {{- end }}
+{{- else }}
+{{- $files := $.Files.Glob (printf "configs/node/%s" $fileName) }}
+{{- $cfg := $files.AsConfig }}
+{{- if and (gt (len $files) 0) (ne $cfg "") }}
+{{ indent 2 $cfg }}
+{{- end }}
+{{- end }}
 {{- end }}

charts/selenium-grid/templates/recorder-configmap.yaml

@@ -21,7 +21,11 @@ data:
 {{ $fileName | indent 2 -}}: |
 {{ $value | indent 4 }}
 {{- else }}
-{{ ($.Files.Glob (printf "configs/recorder/%s" $fileName)).AsConfig | indent 2 }}
+{{- $files := $.Files.Glob (printf "configs/recorder/%s" $fileName) }}
+{{- $cfg := $files.AsConfig }}
+{{- if and (gt (len $files) 0) (ne $cfg "") }}
+{{ indent 2 $cfg }}
+{{- end }}
 {{- end }}
 {{- end }}
 {{- end }}

charts/selenium-grid/templates/secrets.yaml

@@ -42,7 +42,11 @@ data:
 {{- if not (empty $value) }}
   {{ $fileName }}: {{ $value | b64enc }}
 {{- else }}
-{{ ($.Files.Glob (printf "configs/uploader/%s/%s" $.Values.videoRecorder.uploader.name $fileName)).AsSecrets | indent 2 }}
+{{- $files := $.Files.Glob (printf "configs/uploader/%s/%s" $.Values.videoRecorder.uploader.name $fileName) }}
+{{- $cfg := $files.AsSecrets }}
+{{- if and (gt (len $files) 0) (ne $cfg "") }}
+{{ indent 2 $cfg }}
+{{- end }}
 {{- end }}
 {{- end }}
 {{- end }}

charts/selenium-grid/templates/uploader-configmap.yaml

@@ -20,7 +20,11 @@ data:
 {{ $fileName | indent 2 -}}: |
 {{ $value | indent 4 }}
 {{- else }}
-{{ ($.Files.Glob (printf "configs/uploader/%s/%s" $.Values.videoRecorder.uploader.name $fileName)).AsConfig | indent 2 }}
+{{- $files := $.Files.Glob (printf "configs/uploader/%s/%s" $.Values.videoRecorder.uploader.name $fileName) }}
+{{- $cfg := $files.AsConfig }}
+{{- if and (gt (len $files) 0) (ne $cfg "") }}
+{{ indent 2 $cfg }}
+{{- end }}
 {{- end }}
 {{- end }}
 {{- end }}

tests/charts/bootstrap.sh

@@ -13,6 +13,12 @@ python -m pip install pyyaml==6.0.1 \
 
 cd ..
 helm template dummy --values tests/charts/templates/render/dummy.yaml \
+  --set-file 'nodeConfigMap.extraScripts.nodePreStop\.sh=tests/charts/templates/render/dummy_external.sh' \
+  --set-file 'recorderConfigMap.extraScripts.video\.sh=tests/charts/templates/render/dummy_external.sh' \
+  --set-file 'recorderConfigMap.extraScripts.graphQLRecordVideo\.sh=tests/charts/templates/render/dummy_external.sh' \
+  --set-file 'recorderConfigMap.extraScripts.newInsertScript\.sh=tests/charts/templates/render/dummy_external.sh' \
+  --set-file 'uploaderConfigMap.extraScripts.entry_point\.sh=tests/charts/templates/render/dummy_external.sh' \
+  --set-file 'uploaderConfigMap.secretFiles.config\.conf=tests/charts/templates/render/dummy_external.sh' \
   charts/selenium-grid > ./tests/tests/dummy_template_manifests.yaml
 
 python tests/charts/templates/test.py "./tests/tests/dummy_template_manifests.yaml"

tests/charts/ci/auth-ingress-values.yaml

@@ -41,3 +41,31 @@ ingress-nginx:
     kind: DaemonSet
     service:
       type: ClusterIP
+
+videoRecorder:
+  enabled: true
+  uploader:
+    enabled: false
+    name: rclone
+    destinationPrefix: "gs://ndviet"
+    secrets:
+      RCLONE_CONFIG_S3_TYPE: "s3"
+      RCLONE_CONFIG_S3_PROVIDER: "AWS"
+      RCLONE_CONFIG_S3_ENV_AUTH: "true"
+      RCLONE_CONFIG_S3_REGION: "ap-southeast-1"
+      RCLONE_CONFIG_S3_LOCATION_CONSTRAINT: "ap-southeast-1"
+      RCLONE_CONFIG_S3_ACL: "private"
+      RCLONE_CONFIG_S3_ACCESS_KEY_ID: "****"
+      RCLONE_CONFIG_S3_SECRET_ACCESS_KEY: "****"
+      RCLONE_CONFIG_GS_TYPE: "s3"
+      RCLONE_CONFIG_GS_PROVIDER: "GCS"
+      RCLONE_CONFIG_GS_ENV_AUTH: "true"
+      RCLONE_CONFIG_GS_REGION: "asia-southeast1"
+      RCLONE_CONFIG_GS_LOCATION_CONSTRAINT: "asia-southeast1"
+      RCLONE_CONFIG_GS_ACL: "private"
+      RCLONE_CONFIG_GS_ACCESS_KEY_ID: "****"
+      RCLONE_CONFIG_GS_SECRET_ACCESS_KEY: "****"
+      RCLONE_CONFIG_GS_ENDPOINT: "https://storage.googleapis.com"
+      AWS_REGION: "ap-southeast-1"
+      AWS_ACCESS_KEY_ID: "****"
+      AWS_SECRET_ACCESS_KEY: "****"

tests/charts/make/chart_cluster_setup.sh

@@ -39,7 +39,7 @@ echo "Install KEDA core on kind kubernetes cluster"
 kubectl apply --server-side -f https://github.com/kedacore/keda/releases/download/v2.12.1/keda-2.12.1-core.yaml
 
 echo "Load built local Docker Images into Kind Cluster"
-image_list=$(docker images --format "{{.Repository}}:{{.Tag}}" | grep ${NAMESPACE} | grep ${VERSION})
+image_list=$(docker images --format "{{.Repository}}:{{.Tag}}" | grep ${NAMESPACE} | grep ${BUILD_DATE:-$VERSION})
 for image in $image_list; do
     kind load docker-image --name ${CLUSTER_NAME} "$image"
 done

tests/charts/templates/render/dummy_external.sh

@@ -0,0 +1,2 @@
+#!/bin/bash
+echo "Your custom script patch to chart rendering"