introduce profiles adding SingleReplica (SNO) topology on AWS

[mtulio]

Introduce to SingleReplica (SNO) deployment in AWS with a cheaper instance (m6id.xlarge 16GiB RAM, 4vCPU) using ultra-higher disk performance to increase the stability of API, decreasing the resource usage of disk-intensive components, like monitoring stack.

The containers with no persistent storage (ephemeral) may lose the data when the EC2 is stopped (optimized to the dev environment).

The other improvement on the disk layout is to use a dedicated volume for ETCD and increase its stability as it's not sharing IOPS with system and cluster components.

Next step to decrease the costs:

• Support Spot instances to decrease to 1/3 the compute costs: Support Spot Instances on AWS provider #25

[mtulio]

A couple of metrics indicate the low-disk perfoamance and the impact on the API.

• API target group health metric:

• Single Node EBS when running single/shared disk:

[mtulio]

Performance after spliting the disks into different mount points:

$ tail -n 3 sno-run-etcdfio-*.txt
==> sno-run-etcdfio-root.txt <==
--------------------------------------------------------------------------------------------------------------------------------------------------------
INFO: 99th percentile of fsync is 3948544 ns
INFO: 99th percentile of the fsync is within the recommended threshold: - 20 ms, the disk can be used to host etcd

==> sno-run-etcdfio-varlibcontainers.txt <==
--------------------------------------------------------------------------------------------------------------------------------------------------------
INFO: 99th percentile of fsync is 114176 ns
INFO: 99th percentile of the fsync is within the recommended threshold: - 20 ms, the disk can be used to host etcd

==> sno-run-etcdfio-varlibetcd.txt <==
--------------------------------------------------------------------------------------------------------------------------------------------------------
INFO: 99th percentile of fsync is 5996544 ns
INFO: 99th percentile of the fsync is within the recommended threshold: - 20 ms, the disk can be used to host etcd

• Top pods before

$ head -n5 ~/opct/results/opct-sno-aws/info-top-pods-before.txt 
NAMESPACE                                          NAME                                                                    CPU(cores)   MEMORY(bytes)   
openshift-kube-apiserver                           kube-apiserver-ip-10-0-51-31                                            199m         2094Mi          
openshift-monitoring                               prometheus-k8s-0                                                        106m         1900Mi          
openshift-etcd                                     etcd-ip-10-0-51-31                                                      99m          505Mi           
openshift-kube-controller-manager                  kube-controller-manager-ip-10-0-51-31                                   17m          490Mi

• Top pods after (Prometheus dropping the utilization)

$ head -n5 ~/opct/results/opct-sno-aws/sno2-info-top-pods-after.txt
NAMESPACE                                          NAME                                                         CPU(cores)   MEMORY(bytes)   
openshift-kube-apiserver                           kube-apiserver-ip-10-0-49-97                                 222m         1802Mi          
openshift-monitoring                               prometheus-k8s-0                                             39m          865Mi           
openshift-etcd                                     etcd-ip-10-0-49-97                                           70m          400Mi           
openshift-kube-controller-manager                  kube-controller-manager-ip-10-0-49-97                        12m          314Mi