[BACKPORT][v1.6.1][BUG] The feature of auto remount read only volume not work on a single node cluster.

[github-actions]

backport #7843

[longhorn-io-github-bot]

Pre Ready-For-Testing Checklist

• Where is the reproduce steps/test steps documented?
  The reproduce steps/test steps are at:

1. Create single node cluster curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION=v1.28.5+k3s1 sh -s - --write-kubeconfig-mode 644
2. Deploy Longhorn v.1.6.x head
3. Create a single replica volume, attach to node with deployment, record the PVC name
4. Find the PVC mount location mount | grep "/dev/longhorn/${VOLUME_NAME}"
/host/var/lib/kubelet/plugins/kubernetes.io/csi/driver.longhorn.io/${VOLUME_NAME_HASH}/globalmount
5. Remount PVC to read only by below command (remember the path ends with "globalmount" )

mount -o remount,ro /host/var/lib/kubelet/plugins/kubernetes.io/csi/driver.longhorn.io/${VOLUME_NAME_HASH}/globalmount

6. The engine condition should back to FaileSystemReadOnly: false and the IO works in the workload

• PRs:
  • feat(volume): remount volume if it becomes read only (backport #2550) longhorn-manager#2607
  • feat(volume): add remount read only volume grpc (backport #392) longhorn-instance-manager#403

[yangchiu]

Verified passed on v1.6.x-head (longhorn-manager f542e20) following the test plan.