[RLlib] Bug fix: Failed EnvRunners are not restored if there is no local EnvRunner.

[sven1977]

Bug fix: Failed EnvRunners are not restored if there is no local EnvRunner.

The existing logic would restore restarted EnvRunners from the local one.
If there was no local one, the Algorithm.restore_env_runners method would early-out.

The fix is to properly collect the module state from the Learners and local connector pipelines, instead and then sync the newly recreated EnvRunners.

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[Copilot]

Pull Request Overview

This PR fixes a bug where failed EnvRunners were not restored when there was no local EnvRunner available by revising the restoration logic.

• Removed the early return that prevented restoration when both env_runner_group.local_env_runner and self.env_runner were missing.
• Introduced a fallback mechanism that synchronizes EnvRunner state from the LearnerGroup when no local EnvRunner exists.

Comments suppressed due to low confidence (2)

rllib/algorithms/algorithm.py:1888

• The early return was removed to allow fallback restoration. Consider asserting that env_runner_group is always non-null (based on its type hint) to avoid potential errors if an unexpected value is passed.

            A list of EnvRunner indices that have been restored during the call of

rllib/algorithms/algorithm.py:1936

• In the fallback block that syncs state from the learner group, consider adding error handling or an assertion to ensure that 'self.learner_group' is non-null before its usage to avoid potential errors.

            else:

[simonsays1980]

LGTM. I guess the same has to be done for the OfflineEvaluationRunner as well.