initalize gnn network for CIM problem by wesley-stone · Pull Request #60 · microsoft/maro

wesley-stone · 2020-09-22T18:13:07Z

Add a simple solution for the ECR problem:

We use GAT layers to learn the interactions between multi-agents.
We implement the synchronized A2C for multi-processing rollout.

codecov-io · 2020-10-10T00:59:28Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.50%. Comparing base (8d89117) to head (ec6d790).
⚠️ Report is 27 commits behind head on v0.1.

Additional details and impacted files

@@           Coverage Diff           @@
##             v0.1      #60   +/-   ##
=======================================
  Coverage   72.50%   72.50%           
=======================================
  Files          86       86           
  Lines        4062     4062           
=======================================
  Hits         2945     2945           
  Misses       1117     1117

Flag	Coverage Δ
unittests	`72.50% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Jinyu-W

Generally, please read the style guide and go through your PR.

Jinyu-W

Overall, 2 problems are serious:

Please clear the useless code.
Please go through your comments. They should start with a capital letter, end with a period.

Jinyu-W

I think the part of gnn example can be merged into v0.1.

While the change of cim topologies should be merged with the updated naive experiment results (doc).

…#174) * refine readme * feat: refine data push/pull (#138) * feat: refine data push/pull * test: add cli provision testing * fix: style fix * fix: add necessary comments * fix: from code review * add fall back function in weather download (#112) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * added example docs (#136) * added example docs * added citibike greedy example doc * modified citibike doc * fixed PR comments * fixed more PR comments * fixed small formatting issue Co-authored-by: ysqyang <v-yangqi@microsoft.com> * switch the key and value of handler_dict in decorator (#144) * switch the key and value of handler_dict in decorator * add dist decorator UT and fixed multithreading conflict in maro test suite * pr comments update. * resolved comments about decorator UT * rename handler_fun in dist decorator * change self.attr into class_name.attr * update UT tests comments * V0.1 annotation (#147) * refine the annotation of simulator core * remove reward from env(be) * format refined * white spaces test * left-padding spaces refined * format modifed * update the left-padding spaces of docstrings * code format updated * update according to comments * update according to PR comments Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * V0.1 issues in CLI (#157) * fix: add docs for AzCopy * style: refine code style * fix: change stdout format to previous version * test: add dqn tests in grass/k8s mode && add "maro k8s job list" * docs: add scripts for installing AzCopy * v0.1 update citi bike doc and data version (#160) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * update data pipeline docs and data version * update for comments * update for comments * update for lint * update for lint * update for lint * V0.1revert data version to 0.1 (#164) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * update data pipeline docs and data version * update for comments * update for comments * update for lint * update for lint * update for lint * revert data version to 0.1 * initalize gnn network for CIM problem (#60) * initalize gnn netowrk for ECR problem * fix import path error and remove useless function * add the annotation for gnn and state shaper * rename to cim * modify topology 22p * polish coding style in cim.gnn * run pass after polishing * change the name of simplegat to simple_transformer * fix typo and suggestions * fix format * fix code bugs due to refactoring mistake * refine the code format * refine again * refine the format * refine again * refine again * fix syntax bug * refine again * remove useless comments * remove unnecessary blank line * rename pending_event to decision_event * remove some code faced to the future. * change topology * fix bugs in learner: no training logic executed * revert the topology changes Co-authored-by: wenshi-113 <wenshi@microsoft.com> * V0.1 store unpicklable bug fix (#166) * fixed unpicklable store bug * fixed a bug * fixed a bug * fixed a bug * fixed lint formatting * fixed a lint formatting issue * fixed a bug * renamed dump_experience_store to dump_experience_pool * fixed a PR comment Co-authored-by: ysqyang <v-yangqi@microsoft.com> * V0.1 confusing component naming fix (#167) * confusing component naming fix * fixed a bug * fixed a bug * fixed lint formatting issues * fixed a PR comment * replaced 'actor_worker' with 'actor' * fixed a minor formatting issue * fixed a minor formatting issue * fixed a minor formatting issue Co-authored-by: ysqyang <v-yangqi@microsoft.com> * version updated to 0.1.2a0 * modified according to flake8 * update according to lint checker * update import format according to lint checker * update according to lint checker * isort config added * add isort config file path in lint * update import according to lint isort * update import according to lint isort Co-authored-by: Arthur Jiang <sjian@microsoft.com> Co-authored-by: Arthur Jiang <ArthurSJiang@gmail.com> Co-authored-by: Romic Huang <romic.kid@gmail.com> Co-authored-by: zhanyu wang <pocket_2001@163.com> Co-authored-by: ysqyang <ysqyang@gmail.com> Co-authored-by: ysqyang <v-yangqi@microsoft.com> Co-authored-by: kaiqli <59279714+kaiqli@users.noreply.github.com> Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> Co-authored-by: Jinyu Wang <jinywan@microsoft.com> Co-authored-by: WesleyStone <4548088+wesley-stone@users.noreply.github.com> Co-authored-by: wenshi-113 <wenshi@microsoft.com>

* refine readme * feat: refine data push/pull (#138) * feat: refine data push/pull * test: add cli provision testing * fix: style fix * fix: add necessary comments * fix: from code review * add fall back function in weather download (#112) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * added example docs (#136) * added example docs * added citibike greedy example doc * modified citibike doc * fixed PR comments * fixed more PR comments * fixed small formatting issue Co-authored-by: ysqyang <v-yangqi@microsoft.com> * switch the key and value of handler_dict in decorator (#144) * switch the key and value of handler_dict in decorator * add dist decorator UT and fixed multithreading conflict in maro test suite * pr comments update. * resolved comments about decorator UT * rename handler_fun in dist decorator * change self.attr into class_name.attr * update UT tests comments * V0.1 annotation (#147) * refine the annotation of simulator core * remove reward from env(be) * format refined * white spaces test * left-padding spaces refined * format modifed * update the left-padding spaces of docstrings * code format updated * update according to comments * update according to PR comments Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * V0.1 issues in CLI (#157) * fix: add docs for AzCopy * style: refine code style * fix: change stdout format to previous version * test: add dqn tests in grass/k8s mode && add "maro k8s job list" * docs: add scripts for installing AzCopy * v0.1 update citi bike doc and data version (#160) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * update data pipeline docs and data version * update for comments * update for comments * update for lint * update for lint * update for lint * V0.1revert data version to 0.1 (#164) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * update data pipeline docs and data version * update for comments * update for comments * update for lint * update for lint * update for lint * revert data version to 0.1 * initalize gnn network for CIM problem (#60) * initalize gnn netowrk for ECR problem * fix import path error and remove useless function * add the annotation for gnn and state shaper * rename to cim * modify topology 22p * polish coding style in cim.gnn * run pass after polishing * change the name of simplegat to simple_transformer * fix typo and suggestions * fix format * fix code bugs due to refactoring mistake * refine the code format * refine again * refine the format * refine again * refine again * fix syntax bug * refine again * remove useless comments * remove unnecessary blank line * rename pending_event to decision_event * remove some code faced to the future. * change topology * fix bugs in learner: no training logic executed * revert the topology changes Co-authored-by: wenshi-113 <wenshi@microsoft.com> * V0.1 store unpicklable bug fix (#166) * fixed unpicklable store bug * fixed a bug * fixed a bug * fixed a bug * fixed lint formatting * fixed a lint formatting issue * fixed a bug * renamed dump_experience_store to dump_experience_pool * fixed a PR comment Co-authored-by: ysqyang <v-yangqi@microsoft.com> * V0.1 confusing component naming fix (#167) * confusing component naming fix * fixed a bug * fixed a bug * fixed lint formatting issues * fixed a PR comment * replaced 'actor_worker' with 'actor' * fixed a minor formatting issue * fixed a minor formatting issue * fixed a minor formatting issue Co-authored-by: ysqyang <v-yangqi@microsoft.com> * version updated to 0.1.2a0 * modified according to flake8 * update according to lint checker * update import format according to lint checker * update according to lint checker * isort config added * add isort config file path in lint * update import according to lint isort * update import according to lint isort * skip file added for isort linter * filter files option added for isort * V0.1 reformat imports with isort (#184) * init auto-isort * re-format maro init * redis added to isort known third party * update according to lint isort * update according to lint isort * update according to lint * update import path for communication module * update import path for cli scripts module * update according to lint * filter file option added for isort in CONTRIBUTING * update import path for cli utils and communication * update import path for tests * remove filter file option in CONTRIBUTING Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * update according to isort Co-authored-by: Arthur Jiang <sjian@microsoft.com> Co-authored-by: Romic Huang <romic.kid@gmail.com> Co-authored-by: zhanyu wang <pocket_2001@163.com> Co-authored-by: ysqyang <ysqyang@gmail.com> Co-authored-by: ysqyang <v-yangqi@microsoft.com> Co-authored-by: kaiqli <59279714+kaiqli@users.noreply.github.com> Co-authored-by: Jinyu-W <53509467+Jinyu-W@users.noreply.github.com> Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> Co-authored-by: Jinyu Wang <jinywan@microsoft.com> Co-authored-by: WesleyStone <4548088+wesley-stone@users.noreply.github.com> Co-authored-by: wenshi-113 <wenshi@microsoft.com>

initalize gnn netowrk for ECR problem

3b59231

wesley-stone changed the title ~~initalize gnn netowrk for ECR problem~~ initalize gnn network for ECR problem Sep 23, 2020

WenleiShi added 5 commits September 23, 2020 09:59

fix import path error and remove useless function

3370f12

add the annotation for gnn and state shaper

60534e4

Merge branch 'v0.1' into v0.1_feature_ecr_gnn

a39ad4f

rename to cim

882ef26

Merge branch 'v0.1' into v0.1_feature_ecr_gnn

79ff455

wesley-stone changed the title ~~initalize gnn network for ECR problem~~ initalize gnn network for CIM problem Sep 23, 2020

WenleiShi and others added 5 commits September 23, 2020 20:24

modify topology 22p

96f058f

Merge branch 'v0.1' into v0.1_feature_ecr_gnn

f79cebd

Merge branch 'v0.1' into v0.1_feature_ecr_gnn

d8f802a

polish coding style in cim.gnn

44754f9

run pass after polishing

1e78d0b

Jinyu-W suggested changes Sep 28, 2020

View reviewed changes

change the name of simplegat to simple_transformer

686cd33

ArthurJiang assigned wesley-stone Oct 12, 2020

ArthurJiang added the v0.1 label Oct 12, 2020

WenleiShi and others added 2 commits October 12, 2020 15:49

fix typo and suggestions

7c3a9cd

Merge branch 'v0.1' into v0.1_feature_ecr_gnn

35f3b39

Jinyu-W suggested changes Oct 14, 2020

View reviewed changes

wesley-stone requested a review from Jinyu-W October 15, 2020 06:30

WenleiShi added 2 commits October 15, 2020 14:30

fix format

b1a400a

fix code bugs due to refactoring mistake

2f3f071

wesley-stone requested review from Jinyu-W and removed request for Jinyu-W October 15, 2020 08:35

refine the code format

522eeb5

Jinyu-W suggested changes Oct 15, 2020

View reviewed changes

WenleiShi added 2 commits October 15, 2020 17:51

refine again

7da898d

refine the format

f9c779f

WenleiShi added 3 commits October 15, 2020 19:37

refine again

0fd4f5b

refine again

da6fa77

Merge branch 'master' into v0.1_feature_ecr_gnn

53d5584

wesley-stone requested a review from Jinyu-W October 16, 2020 04:34

WenleiShi and others added 3 commits October 16, 2020 15:11

fix syntax bug

40c8047

Merge branch 'v0.1' into v0.1_feature_ecr_gnn

4d7d0e1

refine again

c86d5d9

Jinyu-W suggested changes Oct 16, 2020

View reviewed changes

WenleiShi added 3 commits October 16, 2020 19:36

remove useless comments

ba84802

remove unnecessary blank line

255ed67

rename pending_event to decision_event

c886194

Jinyu-W suggested changes Oct 16, 2020

View reviewed changes

remove some code faced to the future.

669da1a

Jinyu-W reviewed Oct 19, 2020

View reviewed changes

WenleiShi added 4 commits October 22, 2020 18:38

change topology

b207483

fix bugs in learner: no training logic executed

51f9568

revert the topology changes

3804af3

Merge branch 'v0.1' into v0.1_feature_ecr_gnn

ec6d790

wesley-stone requested a review from Jinyu-W October 26, 2020 05:37

Jinyu-W approved these changes Oct 26, 2020

View reviewed changes

Jinyu-W merged commit d00f60f into v0.1 Oct 26, 2020

Jinyu-W deleted the v0.1_feature_ecr_gnn branch October 30, 2020 09:55

Jinyu-W mentioned this pull request Nov 4, 2020

package version updated with several bug fix, refinement, new example #174

Merged

20 tasks

Conversation

wesley-stone commented Sep 22, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-io commented Oct 10, 2020 • edited by codecov Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Jinyu-W left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jinyu-W left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jinyu-W left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

codecov-io commented Oct 10, 2020 •

edited by codecov Bot

Loading