[BACKPORT][v1.4.5][TASK] Review why sessionAffinity: ClientIP is used in most services

[github-actions]

backport #7399

[longhorn-io-github-bot]

Pre Ready-For-Testing Checklist

• Where is the reproduce steps/test steps documented?
  The reproduce steps/test steps are at: [IMPROVEMENT] Remove static sessionAffinity: ClientIP set in most services if not required #7399 (comment).

• Is there a workaround for the issue? If so, where is it documented?
  It a user needs to unset sessionAffinity: ClientIP before the PR, they can make the same manual changes the PR does to deploy/longhorn.yaml or chart/templates. If the user is using a managed chart (e.g. with Rancher), they can make they changes to the deployed services themselves.

• Does the PR include the explanation for the fix or the feature?

• Does the PR include deployment change (YAML/Chart)? If so, where are the PRs for both YAML file and Chart?
  The PR for the YAML change is at: Remove sessionAffinity ClientIP from services (backport #7748) #7766.
  The PR for the chart change is at: Remove sessionAffinity ClientIP from services (backport #7748) #7766.

• Which areas/issues this PR might have potential impacts on?
  It is unclear why sessionAffinity: ClientIP was ever enabled and it does not appear to have any negative effects to disable it. However, there may be an unintended consequence (especially for some corner case deployment) that we do not expect.

[roger-ryao]

Verified on v1.4.x-head 20240206

• longhorn v1.4.x-head ec85f47

The test steps
#7399 (comment)

Result Passed

1. Upgrade Longhorn from version v1.4.4 to v1.4.x-head using the manifest.
2. Upgrade Longhorn from version v1.4.4 to v1.4.x-head using the Helm chart.

In v1.4.4, sessionAffinity: ClientIP was set for services:

kubectl -n longhorn-system get service -oyaml | grep -e "sessionAffinity: ClientIP"
    sessionAffinity: ClientIP
    sessionAffinity: ClientIP
    sessionAffinity: ClientIP
    sessionAffinity: ClientIP

After the upgrade to v1.4.x-head, they no longer exist.