[Feature] Collectors for async envs

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] History.default_spec #2894
• -> [Feature] Collectors for async envs #2893

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2893

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 1 Pending, 2 Unrelated Failures

As of commit 4c9d8a5 with merge base efe9389 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 1.
• Generate documentation / build-docs (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 4707cfdc4e9b418eeeaf95dd06ee8e89cb4a9bb24f1318724da7db3aae07c228 /exec failed with exit code 2
• Habitat Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 1a83eb8f65b8a7d015630314f8262939af8cfebc90c150bd10a33f4c090b07f2 /exec failed with exit code 1
• SOTA Tests on Linux / tests (3.9, 12.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 5c01bca2c40bcfae92c5b2792d77b9083dc92910486270dd9df40bf4752e9a05 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  test/test_env.py::TestLLMEnv::test_llm_from_dataloader_repeats[3-4-False-None-False-None]
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t cbf2682bdfa28ff7bd9d1b9662d5f1e24c88b456ace315fe8350c508b8d7af01 /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Linux / tests-optdeps (3.11, 12.8) / linux-job (gh) (trunk failure)
  test/test_cost.py::TestPPO4LLMs::test_hf[False]
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh) (trunk failure)
  test/test_transforms.py::TestTimer::test_transform_env

This comment was automatically generated by Dr. CI and updates every 15 minutes.