Conformance: Handling optional features

[johnbelamaric]

This is an umbrella issue to track various aspects of managing optional features with respect to Kubernetes conformance.

By the current conformance test requirements, only non-optional features are eligible for conformance tests. However, there are many important features of Kubernetes that are either optional or rely upon an optional component to be deployed into order to function. For example:

• Horizontal pod autoscaling requires the metrics server, an optional component, to be running (Promote pod autoscaling kubernetes#79954).
• Persistent volumes require some underlying CSI provider (PVC/PV conformance testing discussion kubernetes#65155).
• ServiceType LoadBalancer requires some form of external load balancer integration.
• RBAC is an optional feature. Additionally, some features may only be available based upon specific roles (e.g., clusterAdmin).
• DNS is technically optional but currently is required for conformance.

To ensure user workload portability, it is critical to validate that when an optional API / feature is available on a cluster, it behaves in a consistent way across providers. Some related efforts that have attempt to solve this in whole or in part include:

• Validation suites (Add Description of Validation Suites #3361)
• Conformance profiles (SIG-Architecture to do: Conformance test profile definitions #2651)

[johnbelamaric]

/cc @kubernetes/cncf-conformance-wg

[johnbelamaric]

/sig architecture

[johnbelamaric]

/area conformance

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[johnbelamaric]

/remove-lifecycle stale
/lifecycle frozen