Statically assign NodePort service ports

[djtaylor]

Overview

I would like to be able to define the port value for any NodePort service statically. In the case where we integrate with an external service (such as a hardware load balancer) that sends traffic to the services running in a Kube cluster, if we have to rebuild the postgrescluster from scratch, or recreate the service, it assigns a new NodePort, requiring either patching of the service or reconfiguration of the integrating external service.

Use Case

We run two postgresclusters in different Kubernetes cluster in different datacenters, to serve a database to a single service that also runs in both in an HA manner. We pipe traffic to the current active cluster by way of a hardware load balancer, which talks to the clusters via the pgbouncer node port service. If we have to rebuild the clusters, the service gets created fresh with a different NodePort. Right now we are patching it post deployment, but being able to statically set this NodePort value would eliminate the need for this step.

Desired Behavior

When I define that a service should be of type NodePort, I should also be able to statically set the value I want to use for said NodePort.

Environment

• Platform: Kubernetes
• Platform Version: 1.18.18
• PGO Image Tag: ubi8-5.0.4-0
• Postgres Version 13
• Storage: s3
• Number of Postgres clusters: 2

[andrewlecuyer]

@djtaylor thanks so much for the detailed submission.

To get the behavior you desire, i.e. to set a static nodePort, you should be able to simply edit/patch the service after the cluster has been created. For instance, for a PG cluster called hippo I can issue patch the following to explicitly set the nodePort to 30001:

$ kubectl patch svc hippo-ha --type='json' -p='[{"op": "replace", "path": "/spec/ports/0/nodePort", "value":30001}]'
service/hippo-ha patched
$ kubectl get svc hippo-ha
NAME       TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
hippo-ha   NodePort   10.101.137.98   <none>        5432:30001/TCP   117s

And from here the specific setting nodePort should stick, e.g. PGO will not touch that field . For instance, if I change spec.port in the PostgresCluster spec to 5444 I can see that the PG port changes, but my nodePort sticks:

$ kubectl get svc hippo-ha
NAME       TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
hippo-ha   NodePort   10.101.137.98   <none>        5444:30001/TCP   2m29s

Therefore, this should provide as easy way with the current version of PGO to give you the static nodePort behavior you require. Otherwise, we will consider a future enhancement that allows a specific nodePort to be set in the spec.

And lastly, to avoid any unexpected issues, behaviors, etc. when using PGO, I did want to recommend upgrading to a version Kubernertes that is supported by PGO per our Components and Compatibility documentation.

[djtaylor]

@andrewlecuyer Thank you for the response. That is exactly what I am doing, and it is working. Just a side note, I am deploying with Helm/Helmfile, so I think the biggest advantage in being able to do this built in, would be that I wouldn't have to run this step afterwards. That being said, if this can make it into a future release, I would be happy with that.

[cr1cr1]

Actually with pgo 5.0.4 trying to patch any component will not stay for long, as the operator resets any additional configuration within seconds. So far I did not find any parameter or annotation to stop this behavior of strictly enforcing objects.

For example, I cannot set any external IP on the service, the solution being to create a duplicate service not managed by pgo, thus wasting an IP cluster and having 2 objects that do the same thing, only one being actually used.

[andrewlecuyer]

@cr1cr1 since nodePort is not managed by PGO (see the Field Management section in the Kube docs for Server Side Apply for more info), it can be patched/modified and PGO will not touch it. Therefore, the patch to the nodePort in my example above will stay permanently.

However, for other fields in a Service (or any other resource) that are managed by PGO (e.g. service type among many others), PGO does manage those fields and will change them back if they are modified externally. This is again per Server Side Apply conventions in which a controller should force conflicts in order to manage fields within a certain resource or spec: https://kubernetes.io/docs/reference/using-api/server-side-apply/#using-server-side-apply-in-a-controller.

If there are other fields that you would like to modify but are unable to do so, please submit a separate issue.

[cr1cr1]

Thanks for the explanation @andrewlecuyer . I think I understand the general logic about field management.

However is not clear to me the following: for other fields in a Service (or any other resource) that are managed by PGO, if they are managed, shouldn't I be able to set them in the CR for example? But looking at the CRD, seems there is no option to set anything else besides type.

So how are other fields in Service being managed by PGO? It looks like they are actually not considered at all (except type in this case), and of course, as you have said, just enforce whatever is defined in the CRD. So if some fields are not explicitly defined, they will be removed if set by another party or the user. Thus, I cannot set externalIPs field even though it seems not to be managed (unless it is actually managed by inheritance through managing the whole spec of the Service object).
I also assume that x-kubernetes-preserve-unknown-fields: true is not something that you prefer because of the official recommendation :)

[tjmoore4]

@djtaylor following up, we wanted to let you know that this feature is currently in our backlog for implementation in a future PGO release.

Feel free to share any other details about your specific use case that may be helpful as we plan out the feature requirements.