V0.2_refactored_distributed_framework

[ysqyang]

Description

1. Revamp of distributed training framework that consists of: a) Learner as the central driver and coordinator of the training workflow; b) Actor as the simulation data collector and c) ActorProxy as the manager of remote actors for parallel data collection.
2. Added Trajectory abstraction that declares a set of methods to be used as callbacks in actor roll-outs. Scenario-specific logic can be completely encapsulated in the Trajectory abstraction to make the Actor class scenario agnostic.
3. Added OnPolicyLearner and OffPolicyLearner implementations to support common learning paradigms such as actor-critic and DQN.

Linked issue(s)/Pull request(s)

• Issue 171

Type of Change

• Non-breaking bug fix
• Breaking bug fix
• New feature
• Test
• Doc update
• Docker update

Related Component

• Simulation toolkit
• RL toolkit
• Distributed toolkit

Has Been Tested

• OS:
  • Windows
  • Mac OS
  • Linux
• Python version:
  • 3.6
  • 3.7
• Key information snapshot(s):

Needs Follow Up Actions

• New release package
• New docker image

Checklist

• Add/update the related comments
• Add/update the related tests
• Add/update the related documentations
• Update the dependent downstream modules usage

[codecov-io]

Codecov Report

Merging #206 (f658a12) into v0.2 (7c38ffe) will decrease coverage by 0.79%.
The diff coverage is 25.85%.

@@            Coverage Diff             @@
##             v0.2     #206      +/-   ##
==========================================
- Coverage   66.71%   65.92%   -0.80%     
==========================================
  Files         115      115              
  Lines        6024     6098      +74     
==========================================
+ Hits         4019     4020       +1     
- Misses       2005     2078      +73     

Flag	Coverage Δ
unittests	65.92% <25.85%> (-0.80%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
maro/rl/scheduling/scheduler.py	36.84% <0.00%> (+0.47%)	⬆️
maro/rl/scheduling/simple_parameter_scheduler.py	19.14% <0.00%> (ø)
maro/rl/utils/trajectory_utils.py	72.41% <ø> (ø)
maro/utils/exception/error_code.py	100.00% <ø> (ø)
maro/utils/exception/rl_toolkit_exception.py	70.00% <0.00%> (+0.76%)	⬆️
maro/utils/logger.py	64.88% <ø> (ø)
maro/communication/proxy.py	22.69% <8.88%> (-0.41%)	⬇️
maro/rl/agent/dqn.py	22.53% <11.36%> (-1.43%)	⬇️
maro/rl/agent/ddpg.py	25.92% <15.78%> (+2.39%)	⬆️
maro/communication/driver/zmq_driver.py	21.18% <16.66%> (-0.56%)	⬇️
... and 25 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c38ffe...f658a12. Read the comment docs.

[Jinyu-W]

Edit the title and description of this PR @ysqyang