Skip to content

Refine joint decision sequential action mode#219

Merged
Jinyu-W merged 8 commits into
microsoft:v0.2from
chaosddp:v0.2_fix_joint_decision_sequential_action_mode_refine
Feb 1, 2021
Merged

Refine joint decision sequential action mode#219
Jinyu-W merged 8 commits into
microsoft:v0.2from
chaosddp:v0.2_fix_joint_decision_sequential_action_mode_refine

Conversation

@chaosddp
Copy link
Copy Markdown
Contributor

@chaosddp chaosddp commented Dec 20, 2020

Description

Changes:

  1. add a new mode: DecisionModel.JointWithSequentialAction
  2. assign action to pending decision events from beginning to end, ignore events that have no action assigned under joint mode.

Linked issue(s)/Pull request(s)

Type of Change

  • Non-breaking bug fix
  • Breaking bug fix
  • New feature
  • Test
  • Doc update
  • Docker update

Related Component

  • Simulation toolkit
  • RL toolkit
  • Distributed toolkit

Has Been Tested

  • OS:
    • Windows
    • Mac OS
    • Linux
  • Python version:
    • 3.6
    • 3.7
  • Key information snapshot(s):

Needs Follow Up Actions

  • New release package
  • New docker image

Checklist

  • Add/update the related comments
  • Add/update the related tests
  • Add/update the related documentations
  • Update the dependent downstream modules usage

@codecov-io
Copy link
Copy Markdown

codecov-io commented Dec 20, 2020

Codecov Report

Merging #219 (ada572c) into v0.2 (17261e3) will decrease coverage by 0.01%.
The diff coverage is 46.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##             v0.2     #219      +/-   ##
==========================================
- Coverage   64.01%   63.99%   -0.02%     
==========================================
  Files         114      114              
  Lines        5605     5602       -3     
==========================================
- Hits         3588     3585       -3     
  Misses       2017     2017              
Flag Coverage Δ
unittests 63.99% <46.66%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
maro/simulator/core.py 79.38% <42.85%> (-0.62%) ⬇️
maro/simulator/abs_core.py 78.78% <100.00%> (+0.32%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 17261e3...ada572c. Read the comment docs.

@chaosddp chaosddp marked this pull request as ready for review December 20, 2020 05:15
@chaosddp chaosddp requested a review from Jinyu-W January 22, 2021 07:05
@chaosddp chaosddp added awaiting review Wait for the reviewer's comments. in progress Not finished items/bugs. labels Jan 22, 2021
@chaosddp chaosddp removed the in progress Not finished items/bugs. label Jan 25, 2021
Copy link
Copy Markdown
Contributor

@Jinyu-W Jinyu-W left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What situation is this mode for?

Comment thread maro/simulator/core.py
# Ignore following pending events that have no action matched.
pending_event.state = EventState.FINISHED
else:
# Set the state as executing, so event buffer will not pop them again.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pop them again?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as joint mode, that pop all pending decision event at one time, not sequentially, it expect an action list that size is same as decision event. With this change, joint mode can accept actions one by one, and assign the to decision events in the same order, until all decision events get an action.

Comment thread maro/simulator/core.py
for i, pending_event in enumerate(pending_events):
if i >= len(actions):
if self._decision_mode == DecisionMode.Joint:
# Ignore following pending events that have no action matched.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So for the Joint Mode, if we have 5 decision events and receive only 3 actions, we map these 3 actions to the first 3 decision events? and directly mark treat the late 2 decision events as finished without any operation?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, joint mode expect an action list.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And with new mode, we can pop the last 2 decision event again, as we will not mark them as finished.

Comment thread maro/simulator/core.py
pending_events[i].state = EventState.FINISHED
if self._decision_mode == DecisionMode.Sequential:
# Generate a new atom event first.
action_event = self._event_buffer.gen_action_event(self._tick, actions)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for Sequential mode, the actions is a List?

Copy link
Copy Markdown
Contributor Author

@chaosddp chaosddp Feb 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just wrap it as a list, make it same with joint mode, also it can be an action list in sequential mode, we have no requirement at sequential mode.

@Jinyu-W Jinyu-W merged commit b134d1c into microsoft:v0.2 Feb 1, 2021
@chaosddp chaosddp removed the awaiting review Wait for the reviewer's comments. label Mar 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants