Placement enhancement: support spread policy

[qiujian16]

We should be able to support policy in placement:

1. cluster affinity
2. cluster anti affinity
3. spreading

[qiujian16]

/assign @qiujian16
/assign @elgnay

[jnpacker]

To add to the spreading. I was originally interested in putting a NumberOfClusters per predicate, to guarantee each predicate received some clusters, when a subset of all your clusters.

Example:

---
apiVersion: cluster.open-cluster-management.io/v1alpha1
kind: Placement
metadata:
  name: two-cloud-providers
  namespace: default
spec:
  clusterSets:
  - all-clusters
  predicates:
  - requiredClusterSelector:
      numberOfClusters: 3
      labelSelector:
        matchLabels:
          cloud: aws
  - requiredClusterSelector:
      numberOfClusters: 2
      labelSelector:
        matchLabels:
          cloud: gcp

So in this scenario, even if each cloud: type had 6 clusters, placement would return a list of 5 clusters, 3x in AWS and 2x in GCP. If AWS had less then 3 clusters, then as many as are available would be returned. There was also a suggestion to add minNumberOfClusters as a predicate option as well.

[qiujian16]

we were try to build a controller by putting noc in the predicate, but controller cannot easily reason about it. Especially when we add other properties like workload anitaffinity etc...I think we could consider having to placements to achieve this. And set a common label on the placement decision, so consumers can filter all related placementdecision by labels. WDYT @jnpacker @elgnay

[jnpacker]

I see the same thing being achievable if an anti-affinity labelSelector, where I might use the label key or keys, like cloud.

And so if Spec.NumberOfClusters is 4 and I have two distinct values of label key cloud then I would spread out my results from the two, and is consumed in a single predicate.

[qiujian16]

This is current in progress document https://docs.google.com/document/d/1aaRSnyChczgJvejzug-fkY2eYhhQWfa2CPmqRq5s-SA/edit#heading=h.t2wrspluvypn

I think what you mention according to the doc could be done by

spreadPolicy:
   spreadConstraints:
   - maxSkew: 1
     whenUnsatisfiable: DoNotSelect
     topologyKey: region.open-cluster-management.io
     topologyKeyType: Claim

so if we want 4 clusters only from aws and gcp and we want them to spread to them. It will be like

apiVersion: cluster.open-cluster-management.io/v1alpha1
kind: Placement
metadata:
 name: ocp-placement
spec:
  numberOfClusters: 4
  predicates:
    requiredClusterSelector:
      claimSelector:
         matchExpressions:
           claimSelector:
             matchExpressions:
             - key: platform.open-cluster-management.io
               operator: In
               values:
               - AWS
               - GCP
 spreadPolicy:
   spreadConstraints:
   - maxSkew: 1
     whenUnsatisfiable: DoNotSelect
     topologyKey: platform.open-cluster-management.io
     topologyKeyType: Claim

[qiujian16]

/kind feature

[haoqing0110]

The proposal about Spread policy in Placement is merged.

[qiujian16]

/close

[openshift-ci]

@qiujian16: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.