[Trainers] SAC Trainer and algorithms

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] Async collection within trainers #3173
• -> [Trainers] SAC Trainer and algorithms #3172
• [Feature] Collectors' getattr_policy and getattr_env #3171
• [Feature] track_policy_version in collectors.py #3170
• [Typing] Edit wrongfully set str type annotations #3169

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3172

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 9 New Failures, 1 Cancelled Job, 2 Unrelated Failures

As of commit 1131273 with merge base f8bb71b ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t f4c77f1c4f4cf1e4393d40687317a70bd6a167a8885c3903cf1bb1dfb746c703 /exec failed with exit code 1
• Lint / python-source-and-configs / linux-job (gh)
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  test/test_configs.py::TestDataConfigs::test_replay_buffer_config
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_configs.py::TestDataConfigs::test_replay_buffer_config
• Unit-tests on Linux / tests-cpu (3.12) / linux-job (gh)
  test/test_configs.py::TestDataConfigs::test_replay_buffer_config
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  test/test_env.py::TestNonTensorEnv::test_parallel[False-False]
• Unit-tests on Linux / tests-gpu (3.11, 12.8) / linux-job (gh)
  test/test_configs.py::TestDataConfigs::test_replay_buffer_config
• Unit-tests on Linux / tests-stable-gpu (3.10, 11.8) / linux-job (gh)
  test/test_configs.py::TestDataConfigs::test_replay_buffer_config

CANCELLED JOB - The following job was cancelled. Please retry:

• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Libs Tests on Linux / unittests-gym (3.9, 12.8) / linux-job (gh) (trunk failure)
  test/test_libs.py::TestGym::test_gym_fake_td[True-False-3-HalfCheetah-v2]
• LLM Tests on Linux / unittests (3.9, 12.8) / linux-job (gh) (trunk failure)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.