Tolerations and affinity

[totycro]

This PR adds tolerations and affinity settings for all pods. This is necessary for clusters with multiple node types, where the operators want to run eoapi on specific ones.

I've tested the change with the following values. With this all pods run on the desired nodes (except for pgbounce and pgbackuprest, but these can be configured by using postgrescluster values).

The values are a bit repetitive, but it might be a realistic use case that e.g. titiler should run on specific nodes, which would not be possible with a global tolerations/affinity setting (although it would be enough for my use case for now).

I'm not sure if these settings can realistically be tested in CI in k3s. We'd need to have different nodes in k3s and make sure that the services are spawned in the correct ones.

Related discussion: #172

stac:
  settings:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: "my-node-key"
              operator: In
              values:
              - user
    tolerations:
      - key: "my-taint-name"
        operator: "Equal"
        value: "user"
        effect: "NoSchedule"

raster:
  settings:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: "my-node-key"
              operator: In
              values:
              - user
    tolerations:
      - key: "my-taint-name"
        operator: "Equal"
        value: "user"
        effect: "NoSchedule"

vector:
  settings:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: "my-node-key"
              operator: In
              values:
              - user
    tolerations:
      - key: "my-taint-name"
        operator: "Equal"
        value: "user"
        effect: "NoSchedule"

docServer:
  settings:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: "my-node-key"
              operator: In
              values:
              - user
    tolerations:
      - key: "my-taint-name"
        operator: "Equal"
        value: "user"
        effect: "NoSchedule"

pgstacBootstrap:
  settings:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: "my-node-key"
              operator: In
              values:
              - user
    tolerations:
      - key: "my-taint-name"
        operator: "Equal"
        value: "user"
        effect: "NoSchedule"

postgrescluster:
  instances:
  - name: eoapi
    replicas: 1
    affinity: 
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
          - matchExpressions:
            - key: "my-node-key"
              operator: In
              values:
              - user

    tolerations: 
    - key: "my-taint-name"
      operator: "Equal"
      value: "user"
      effect: "NoSchedule"

    dataVolumeClaimSpec:
      accessModes:
      - "ReadWriteOnce"
      resources:
        requests:
          storage: "10Gi"
          cpu: "1024m"
          memory: "3048Mi"

[pantierra]

Many thanks for the PR! Looks good, already. I was just wondering whether we could avoid setting empty values for affinity and tolerations in the values.yaml. I'd prefer to have them added only if needed.

[totycro]

Many thanks for the PR! Looks good, already. I was just wondering whether we could avoid setting empty values for affinity and tolerations in the values.yaml. I'd prefer to have them added only if needed.

Thanks for the review @pantierra!

I've removed the empty values from values.yaml as you have suggested. I've added them because then they have more visibility, but it would also work if they are mentioned in somewhere else.

Btw I used the with syntax because it has in implicit if, so no output would be generated if the affinity/tolerations are not present or an empty list or an empty object. Also then you only need to specify the key once since it updates the dot to refer to the value.
Is there a particular reason why you prefer an explicit if here? It would mean a bit less helm template magic and more consistency with the rest.

[pantierra]

No, all good. with is just fine then!