[RLlib] Fix time dimension shaping for PyTorch RNN models.

[XuehaiPan]

Why are these changes needed?

• Fix tensor.reshape to tensor.transpose when storing data in time-major format. See [RLlib] Bug when using RNNs, time_major and PyTorch #13075 (comment) for more details.
• Use tensor.view instead of tensor.reshape, which will not create a new tensor instance.

Related issue number

Closes #13075

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[bveeramani]

‼️ ACTION REQUIRED ‼️

We've switched our code formatter from YAPF to Black (see #21311).

To prevent issues with merging your code, here's what you'll need to do:

1. Install Black

pip install -I black==21.12b0

2. Format changed files with Black

curl -o format-changed.sh https://gist.githubusercontent.com/bveeramani/42ef0e9e387b755a8a735b084af976f2/raw/7631276790765d555c423b8db2b679fd957b984a/format-changed.sh
chmod +x ./format-changed.sh
./format-changed.sh
rm format-changed.sh

3. Commit your changes.

git add --all
git commit -m "Format Python code with Black"

4. Merge master into your branch.

git pull upstream master

5. Resolve merge conflicts (if necessary).

After running these steps, you'll have the updated format.sh.

[gjoliver]

would be nice to create a small unit test for add_time_dimension() based on your research.
good debugging btw.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[sven1977]

Hey @XuehaiPan , thanks a ton for this PR! Could you check the failing test cases here?
https://buildkite.com/ray-project/ray-builders-pr/builds/29015

Some things may have been broken by your changes.

[XuehaiPan]

Hi @sven1977, I updated my PR. The CI test errors seem irrelevant.

[sven1977]

Merged! :) Thanks for the fix @XuehaiPan.