[RLlib; docs] Fix docstring example for custom MultiRLModule with shared encoder.

[sven1977]

Fix docstring example for custom MultiRLModule with shared encoder.

• Needed action_space definition on shared encoder module.
• do not require action_space to be defined in RLModuleSpec when build is called. The spec may be for a component that doesn't care about that space (like a shared encoder or a value function).

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[Copilot]

Pull Request Overview

This PR fixes the docstring example for a custom MultiRLModule with a shared encoder by ensuring that a required action_space is defined and the correct module is referenced in the example.

• Added action_space to the shared encoder module specification.
• Updated the module retrieval call to explicitly reference the "p0" module.
• Modified algorithm configuration to fallback to policy_spec for missing observation or action spaces.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
rllib/examples/rl_modules/classes/vpg_using_shared_encoder_rlm.py	Added action_space to shared encoder's spec and updated a module retrieval call to get_module("p0").
rllib/algorithms/algorithm_config.py	Revised fallback logic to set missing observation_space or action_space from policy_spec.

Comments suppressed due to low confidence (1)

rllib/algorithms/algorithm_config.py:4641

• [nitpick] Consider adding an inline comment explaining the fallback logic for setting module_spec observation_space and action_space from policy_spec, to aid future maintainers.

            ):

[Copilot]

Using an explicit module identifier 'p0' clarifies which module to print. Please ensure that 'p0' is consistently defined in your module specifications.