MAKE THIS REPO BETTER

MARLWARE TODO LIST

0. Add actors [ high priority ]

• Implement actor - @ray.remote

• Move actor's common parts into actor proto

• ray compatible

• memory parallel collecting is a must, compute is next step

• worker should be remote ( ray actor ) - wrapper

• it should take ref to the cortex model ( ray.put(cortex) )

• it should take N different instances of environment - can be copied

• replay buffer should be a single instance that stores N collected shards of data from actors

• answer the question about the seed: cortex | policy | env seed. What will happen with const seed?

• env should be remote ( ray actor ) - wrapper

• refactor SC2 environment class together with hydra stuff

• parameters server: https://www.youtube.com/watch?v=XjmbKkc98MA ( 1:09:30 )

1. Rewrite memory [ high priority ]

• get rid of inheritance
• replay memory holds rollout trajectories
• ray compatible

2. Refactor Cortex

• extract common stuff into core
• implement child node that derives from Cortex protocol

3. Add worker

• Implement worker which job is to collect data
• Move Worker's common parts into worker port
• ray compatible

4. Implement new Tuner mechanizm

• Implement new Tuner mechanizm for ray

5. Connect to ray

• self explanatory

6. Fix docker

• make this work
• the issue with docker was raised in the pymarl repo: https://github.com/oxwhirl/pymarl/issues/157

7. Rewrite evaluator

• Boilerplate code
• Bad logging
• Overall mess

8. Rewrite logger

• Overall mess

9. Migrate to SMACV2

• Add SMACV2 support
• Link: https://github.com/oxwhirl/smacv2

10. Add GPU and MPS acceleration

• once ray is integrated make sure MPS and GPU can be utilised for compute

11. Move away from logging - Loguru

• migrate to Loguru from logging
• link to loguru: https://github.com/Delgan/loguru