Introduce option to select storage tiers

[kishen-v]

Motivation:
To enable tier-selection of backing storage devices that provide higher bandwidth for perf-tests:

Current CI:
I0129 10:35:28.869202   27891 shared_informer.go:320] Caches are synced for PodsIndexer
I0129 10:36:35.884296   27891 phase_latency.go:146] PodStartupLatency: perc50: 2.098175336s, perc90: 2.415289209s, perc99: 3.476953133s
I0129 10:36:35.884370   27891 phase_latency.go:146] PodStartupLatency: perc50: 2.098175336s, perc90: 2.415289209s, perc99: 3.476953133s; threshold 10s
I0129 10:36:35.884399   27891 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0129 10:36:35.884451   27891 phase_latency.go:146] PodStartupLatency: perc50: 1s, perc90: 1s, perc99: 2s
I0129 10:36:35.884549   27891 phase_latency.go:146] PodStartupLatency: perc50: 1.172445589s, perc90: 1.405423512s, perc99: 1.992513888s
I0129 10:36:35.884773   27891 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0129 10:36:35.884827   27891 phase_latency.go:146] PodStartupLatency: perc50: 1s, perc90: 1s, perc99: 2s
I0129 10:36:35.884898   27891 phase_latency.go:146] PodStartupLatency: perc50: 1.172445589s, perc90: 1.405423512s, perc99: 1.992513888s
I0129 10:36:35.884967   27891 phase_latency.go:146] PodStartupLatency: perc50: 2.098175336s, perc90: 2.415289209s, perc99: 3.476953133s
I0129 10:36:35.885043   27891 phase_latency.go:146] PodStartupLatency: perc50: 2.098175336s, perc90: 2.415289209s, perc99: 3.476953133s; threshold 10s
I0129 10:36:35.885123   27891 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0129 10:36:35.885170   27891 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0129 10:36:35.885212   27891 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0129 10:36:35.885257   27891 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0129 10:36:35.885304   27891 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s; threshold 10s
W0129 10:36:36.145632   27891 prometheus_measurement.go:68] APIResponsivenessPrometheusSimple: Prometheus is disabled, skipping the measurement!

With tier0:
W0131 05:47:09.687976  101431 prometheus_measurement.go:68] APIResponsivenessPrometheusSimple: Prometheus is disabled, skipping the measurement!
I0131 05:47:09.848050  101431 shared_informer.go:313] Waiting for caches to sync for PodsIndexer
I0131 05:47:10.448791  101431 shared_informer.go:320] Caches are synced for PodsIndexer
I0131 05:48:01.404649  101431 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0131 05:48:01.404831  101431 phase_latency.go:146] PodStartupLatency: perc50: 1s, perc90: 1s, perc99: 1s
I0131 05:48:01.404980  101431 phase_latency.go:146] PodStartupLatency: perc50: 1.0697256s, perc90: 1.37830339s, perc99: 2.00765719s
I0131 05:48:01.405140  101431 phase_latency.go:146] PodStartupLatency: perc50: 1.966059912s, perc90: 2.293791977s, perc99: 2.417418868s
I0131 05:48:01.405313  101431 phase_latency.go:146] PodStartupLatency: perc50: 1.966059912s, perc90: 2.293791977s, perc99: 2.417418868s; threshold 5s
I0131 05:48:01.405810  101431 phase_latency.go:146] PodStartupLatency: perc50: 1s, perc90: 1s, perc99: 1s
I0131 05:48:01.406227  101431 phase_latency.go:146] PodStartupLatency: perc50: 1.0697256s, perc90: 1.37830339s, perc99: 2.00765719s
I0131 05:48:01.407824  101431 phase_latency.go:146] PodStartupLatency: perc50: 1.966059912s, perc90: 2.293791977s, perc99: 2.417418868s
I0131 05:48:01.407999  101431 phase_latency.go:146] PodStartupLatency: perc50: 1.966059912s, perc90: 2.293791977s, perc99: 2.417418868s; threshold 5s
I0131 05:48:01.408105  101431 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0131 05:48:01.408385  101431 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0131 05:48:01.408557  101431 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0131 05:48:01.408677  101431 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s; threshold 5s
I0131 05:48:01.408798  101431 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
I0131 05:48:01.408910  101431 phase_latency.go:146] PodStartupLatency: perc50: 0s, perc90: 0s, perc99: 0s
W0131 05:48:01.467663  101431 prometheus_measurement.go:68] APIResponsivenessPrometheusSimple: Prometheus is disabled, skipping the measurement!

A delta of approx. ~1 second improvement is observed for disks with higher bandwidth, which are provisioned with tier0 compared with the default tier1 storage tier.

Todos:
To revert: http://github.com/ppc64le-cloud/test-infra/pull/430

[mkumatag]

@kishen-v @Rajalakshmi-Girish ping ^^

[Rajalakshmi-Girish]

@mkumatag the --extra-vars flag that we added, passes values to the ansible playbook. We cannot use it in .tf files as of now.

Looks like we must modify the code around https://github.com/ppc64le-cloud/kubetest2-plugins/blob/master/kubetest2-tf/deployer/deployer.go#L251 to use that flag in more generic way.

[mkumatag]

@mkumatag the --extra-vars flag that we added, passes values to the ansible playbook. We cannot use it in .tf files as of now.

Looks like we must modify the code around https://github.com/ppc64le-cloud/kubetest2-plugins/blob/master/kubetest2-tf/deployer/deployer.go#L251 to use that flag in more generic way.

I think terraform by default supports this feature, @kishen-v can you try this solution mentioned here in this link - https://stackoverflow.com/questions/52774464/does-terraform-allow-us-to-override-variables

[kishen-v]

Hi @mkumatag @Rajalakshmi-Girish,
I had tried though by exporting a value to through TF_VAR_storage_tier, it worked as intended.

export TF_VAR_powervs_storage_tier=tier0

  # module.master.ibm_pi_instance.pvminstance[0] will be created
  + resource "ibm_pi_instance" "pvminstance" {
  ...
  ...
      + pi_storage_pool_affinity       = true
      + pi_storage_type                = "tier0"
      + pi_sys_type                    = "s922"

Thanks!

[mkumatag]

I had tried though by exporting a value to through TF_VAR_storage_tier, it worked as intended.

can we close this PR then?

[mkumatag]

/lgtm

[ppc64le-cloud-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kishen-v, mkumatag

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [mkumatag]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment