Clean dangling DHCP Options (#668)

* Clean dangling DHCP Options

Signed-off-by: Michal Hajas <mhajas@redhat.com>

* making the script not to run with automation and adding docs

Signed-off-by: Kamesh Akella <kamesh.asp@gmail.com>

* removing outdated warning

Signed-off-by: Kamesh Akella <kamesh.asp@gmail.com>

* moving the docs to util instead

Signed-off-by: Kamesh Akella <kamesh.asp@gmail.com>

---------

Signed-off-by: Michal Hajas <mhajas@redhat.com>
Signed-off-by: Kamesh Akella <kamesh.asp@gmail.com>
Co-authored-by: Kamesh Akella <kamesh.asp@gmail.com>
Co-authored-by: Kamesh Akella <kakella@redhat.com>

doc/kubernetes/modules/ROOT/pages/openshift/cross-site-rosa.adoc

@@ -77,7 +77,6 @@ To create a deployment for specific Keycloak source code, git repository and bra
 This replaces deployments for both Keycloak and Keycloak operator.
 Specify the following variables for using custom source code:
 
-
 |===
 |Variable |Details
 
@@ -99,3 +98,7 @@ AWS provides a https://github.com/awslabs/aws-advanced-jdbc-wrapper[JDBC driver
 To disable the AWS JDBC driver, set the `KC_USE_AWS_JDBC_WRAPPER` variable to `false`.
 
 To specify the version of the AWS JDBC driver, set the `KC_AWS_JDBC_WRAPPER_URL` variable to the URL of corresponding jar file.
+
+=== Warnings / Known issues
+
+* We know that sometimes during the ROSA cluster creation, few orphaned DHCP options sets are created. We can monitor and clean them up using a bash script, for more info on that see, how to xref:util/clean-orphan-dhcp-options.adoc[].

doc/kubernetes/modules/ROOT/pages/util/clean-orphan-dhcp-options.adoc

@@ -0,0 +1,5 @@
+= Clean the orphaned DHCP options from a ROSA cluster creation
+
+This is a workaround for the following issue https://issues.redhat.com/browse/OCPBUGS-1838.
+
+We can use the link:{github-files}/provision/aws/rosa_clean_aws_dhcp_options.sh[rosa_clean_aws_dhcp_options.sh] script, to look for dangling DHCP Options Sets and delete them if need be.

provision/aws/rosa_clean_aws_dhcp_options.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+if [[ "$RUNNER_DEBUG" == "1" ]]; then
+  set -x
+fi
+
+for REG in $(aws account list-regions --query 'Regions[?RegionOptStatus != `DISABLED`].RegionName' --output text);
+do
+  # Get all DHCP options that are tagged with red-hat-clustertype=rosa
+  DHCP_OPTIONS_JSON=$(aws ec2 describe-dhcp-options --region "$REG" --filters Name=tag:red-hat-clustertype,Values=rosa  --query 'DhcpOptions[*]' --output json --no-cli-pager 2>/dev/null)
+
+  # Iterate over all DHCP options
+  DHCP_OPTIONS_IDS=( $(echo $DHCP_OPTIONS_JSON | jq -r '.[].DhcpOptionsId') )
+  if [ ${#DHCP_OPTIONS_IDS[@]} -gt 0 ]; then
+    echo "$REG region contains DHCP options that were not cleaned up [${#DHCP_OPTIONS_IDS[@]}]"
+    for DHCP_OPTIONS_ID in "${DHCP_OPTIONS_IDS[@]}"; do
+      # All ROSA resources are tagged with "kubernetes.io/cluster/<cluster-name>"="owned" therefore we can use this to find the VPC
+
+      # Get the tag key and value from the found DHCP options
+      VPC_TAG_KEY=$(echo $DHCP_OPTIONS_JSON | jq -r ".[] | select(.DhcpOptionsId == \"$DHCP_OPTIONS_ID\") | .Tags[] | select(.Key | startswith(\"kubernetes.io/cluster/\")) | .Key")
+      VPC_TAG_VALUE=$(echo $DHCP_OPTIONS_JSON | jq -r ".[] | select(.DhcpOptionsId == \"$DHCP_OPTIONS_ID\") | .Tags[] | select(.Key | startswith(\"kubernetes.io/cluster/\")) | .Value")
+
+      # Find VPC based on the tag and value matching the DHCP options
+      VPC_ID=$(aws ec2 describe-vpcs --region "$REG" --filters Name=tag:"$VPC_TAG_KEY",Values="$VPC_TAG_VALUE" --query 'Vpcs[*].VpcId' --output text --no-cli-pager 2>/dev/null)
+
+      # If no VPC was found, delete the DHCP options
+      if [ -z "$VPC_ID" ]; then
+        echo "Deleting DHCP options $DHCP_OPTIONS_ID as no VPC was found"
+        aws ec2 delete-dhcp-options --region "$REG" --dhcp-options-id "$DHCP_OPTIONS_ID" --no-cli-pager
+      fi
+    done
+  fi
+done
+

provision/aws/rosa_cluster_reaper.sh

@@ -32,3 +32,5 @@ echo "Finished reaping all possible clusters at $(date -uIs)"
 
 
 
+
+