Consider a node with a failed reusable replica as still used (backport #2650)

[mergify]

This is an automatic backport of pull request #2650 done by Mergify.
Cherry-pick of 29a895c has failed:

On branch mergify/bp/v1.5.x/pr-2650
Your branch is up to date with 'origin/v1.5.x'.

You are currently cherry-picking commit 29a895c2.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   scheduler/replica_scheduler.go
	both modified:   scheduler/replica_scheduler_test.go

no changes added to commit (use "git add" and/or "git commit -a")

Cherry-pick of 65ceb37 has failed:

On branch mergify/bp/v1.5.x/pr-2650
Your branch is ahead of 'origin/v1.5.x' by 1 commit.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 65ceb37c.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   scheduler/replica_scheduler.go
	both modified:   scheduler/replica_scheduler_test.go

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

---

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

• @Mergifyio refresh will re-evaluate the rules
• @Mergifyio rebase will rebase this PR on its base branch
• @Mergifyio update will merge the base branch into this PR
• @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

• look at your merge queues
• generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

[mergify]

Cherry-pick of 29a895c has failed:

On branch mergify/bp/v1.5.x/pr-2650
Your branch is up to date with 'origin/v1.5.x'.

You are currently cherry-picking commit 29a895c2.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   scheduler/replica_scheduler.go
	both modified:   scheduler/replica_scheduler_test.go

no changes added to commit (use "git add" and/or "git commit -a")

Cherry-pick of 65ceb37 has failed:

On branch mergify/bp/v1.5.x/pr-2650
Your branch is ahead of 'origin/v1.5.x' by 1 commit.
  (use "git push" to publish your local commits)

You are currently cherry-picking commit 65ceb37c.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   scheduler/replica_scheduler.go
	both modified:   scheduler/replica_scheduler_test.go

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

[ejweber]

I had to change this somewhat significantly since v1.5.x there was a scheduler refactor that was not backported to v1.5.x. Running the full suite of end-to-end tests against it before requesting review.

https://ci.longhorn.io/job/private/job/longhorn-tests-regression/6603/

[ejweber]

Actual test run (the previous was set up incorrectly):

https://ci.longhorn.io/job/private/job/longhorn-tests-regression/6614/testReport/