package version updated with several bug fix, refinement, new example by Jinyu-W · Pull Request #174 · microsoft/maro

Jinyu-W · 2020-10-30T10:03:09Z

Description

This PR contains several PRs including some bug fix, new example addition, doc addition and refinement, new features, new tests. The package version is also updated from 0.1.1a11 to 0.1.2a0.

Linked issue(s)/Pull request(s)

138:
- refine data push/pull in cli
112:
- add fall back function in weather download
136:
- add docs for example
144:
- switching the key and values of the handler dict which is used in the dist decorator
- add unit tests for the dist decorator
- fixed the multithreading deadlock in the maro unit test suite
147:
- remove useless reward() from Env, BE, yaml config
- refine the format & style for core & BE
157:
- add docs for AzCopy
- add dqn tests in grass/k8s mode
160, 164:
- fix the data generation description for citi bike in docs
60:
- add example/cim/gnn
166:
- fix the uppicklable bug in RL toolkit store
167:
- rename the confusing components in RL toolkit and example/cim/dqn

Type of Change

Related Component

Simulation toolkit
RL toolkit
Distributed toolkit

Has Been Tested

OS:
- Windows
- Mac OS
- Linux
Python version:
- 3.6
- 3.7
Key information snapshot(s):

Needs Follow Up Actions

New release package
New docker image

Checklist

Add/update the related comments
Add/update the related tests
Add/update the related documentations
Update the dependent downstream modules usage

* feat: refine data push/pull * test: add cli provision testing * fix: style fix * fix: add necessary comments * fix: from code review

* fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check

* added example docs * added citibike greedy example doc * modified citibike doc * fixed PR comments * fixed more PR comments * fixed small formatting issue Co-authored-by: ysqyang <v-yangqi@microsoft.com>

* switch the key and value of handler_dict in decorator * add dist decorator UT and fixed multithreading conflict in maro test suite * pr comments update. * resolved comments about decorator UT * rename handler_fun in dist decorator * change self.attr into class_name.attr * update UT tests comments

* refine the annotation of simulator core * remove reward from env(be) * format refined * white spaces test * left-padding spaces refined * format modifed * update the left-padding spaces of docstrings * code format updated * update according to comments * update according to PR comments Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com>

* fix: add docs for AzCopy * style: refine code style * fix: change stdout format to previous version * test: add dqn tests in grass/k8s mode && add "maro k8s job list" * docs: add scripts for installing AzCopy

* fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * update data pipeline docs and data version * update for comments * update for comments * update for lint * update for lint * update for lint

* fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * add noaa weather source * fix weather reset and weather comment * add comment for weather data url * some format update * add fall back function in weather download * update comment * update for comments * update comment * add period * fix for pylint * update for pylint check * update data pipeline docs and data version * update for comments * update for comments * update for lint * update for lint * update for lint * revert data version to 0.1

* initalize gnn netowrk for ECR problem * fix import path error and remove useless function * add the annotation for gnn and state shaper * rename to cim * modify topology 22p * polish coding style in cim.gnn * run pass after polishing * change the name of simplegat to simple_transformer * fix typo and suggestions * fix format * fix code bugs due to refactoring mistake * refine the code format * refine again * refine the format * refine again * refine again * fix syntax bug * refine again * remove useless comments * remove unnecessary blank line * rename pending_event to decision_event * remove some code faced to the future. * change topology * fix bugs in learner: no training logic executed * revert the topology changes Co-authored-by: wenshi-113 <wenshi@microsoft.com>

* fixed unpicklable store bug * fixed a bug * fixed a bug * fixed a bug * fixed lint formatting * fixed a lint formatting issue * fixed a bug * renamed dump_experience_store to dump_experience_pool * fixed a PR comment Co-authored-by: ysqyang <v-yangqi@microsoft.com>

* confusing component naming fix * fixed a bug * fixed a bug * fixed lint formatting issues * fixed a PR comment * replaced 'actor_worker' with 'actor' * fixed a minor formatting issue * fixed a minor formatting issue * fixed a minor formatting issue Co-authored-by: ysqyang <v-yangqi@microsoft.com>

ArthurJiang · 2020-11-04T06:36:09Z

Add the description for details.

ArthurJiang · 2020-11-04T06:37:10Z

change a meaningful title

ArthurJiang · 2020-11-04T06:39:23Z

update version to 0.1.2a0 in maro/misc.py

Jinyu-W · 2020-11-04T07:24:36Z

Add the description for details.

done

Jinyu-W · 2020-11-04T07:24:51Z

update version to 0.1.2a0 in maro/misc.py

done

Jinyu-W · 2020-11-04T07:27:56Z

change a meaningful title

done

codecov-io · 2020-11-04T08:50:06Z

Codecov Report

Merging #174 into master will decrease coverage by 0.01%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master     #174      +/-   ##
==========================================
- Coverage   72.50%   72.49%   -0.02%     
==========================================
  Files          86       86              
  Lines        4062     4064       +2     
==========================================
+ Hits         2945     2946       +1     
- Misses       1117     1118       +1

Flag	Coverage Δ
unittests	`72.49% <50.00%> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...topologies/single_learner_multi_actor_sync_mode.py	`36.17% <36.36%> (ø)`
maro/rl/agent/abs_agent.py	`51.51% <50.00%> (-1.61%)`	⬇️
maro/rl/storage/column_based_store.py	`65.95% <50.00%> (-1.83%)`	⬇️
maro/rl/learner/simple_learner.py	`35.71% <66.66%> (+3.45%)`	⬆️
maro/utils/logger.py	`44.53% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update df94746...15883b3. Read the comment docs.

Arthur Jiang and others added 25 commits October 1, 2020 15:08

refine readme

95c6413

Merge branch 'v0.1' of https://github.com/microsoft/maro into v0.1

ddd15a5

Merge branch 'master' into v0.1

de6f89d

feat: refine data push/pull (#138)

b3c01ca

* feat: refine data push/pull * test: add cli provision testing * fix: style fix * fix: add necessary comments * fix: from code review

Merge branch 'master' into v0.1

080714c

Merge branch 'master' into v0.1

1fa9854

Merge branch 'master' into v0.1

ef7f870

added example docs (#136)

5bfc5eb

* added example docs * added citibike greedy example doc * modified citibike doc * fixed PR comments * fixed more PR comments * fixed small formatting issue Co-authored-by: ysqyang <v-yangqi@microsoft.com>

Merge branch 'master' into v0.1

11714ea

Merge branch 'v0.1' of https://github.com/microsoft/maro into v0.1

71c7e5b

Merge branch 'master' into v0.1

1c9e60d

Merge branch 'master' into v0.1

0b11548

Merge branch 'master' into v0.1

e2c41a4

Merge branch 'master' into v0.1

c50ef92

Merge branch 'master' into v0.1

2d394bd

V0.1 issues in CLI (#157)

6b667fc

* fix: add docs for AzCopy * style: refine code style * fix: change stdout format to previous version * test: add dqn tests in grass/k8s mode && add "maro k8s job list" * docs: add scripts for installing AzCopy

Merge branch 'master' into v0.1

8e6a814

Jinyu-W requested a review from ArthurJiang October 30, 2020 10:03

Jinyu-W assigned wesley-stone, romickid, ysqyang and Jinyu-W Oct 30, 2020

Jinyu-W assigned wesley-stone, romickid, ysqyang, Jinyu-W and kaiqli and unassigned Jinyu-W and kaiqli Oct 30, 2020

Jinyu-W added awaiting review Wait for the reviewer's comments. v0.1 labels Oct 30, 2020

version updated to 0.1.2a0

c579f95

Jinyu-W changed the title ~~merge V0.1 into master~~ package version updated with several bug fix, refinement, new example Nov 4, 2020

Jinyu Wang added 4 commits November 4, 2020 08:08

modified according to flake8

e2dfd2d

update according to flake8

3f4c70d

update according to lint checker

fdff7ea

update import format according to lint checker

5fb4e5f

Jinyu Wang added 6 commits November 4, 2020 09:07

update according to lint checker

e6b9ad0

isort config added

3f39c72

add isort config file path in lint

15883b3

Merge branch 'master' into v0.1

7d6fcb5

update import according to lint isort

4639617

update import according to lint isort

9c1bea7

ArthurJiang approved these changes Nov 5, 2020

View reviewed changes

ArthurJiang merged commit 4777d95 into master Nov 5, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

package version updated with several bug fix, refinement, new example#174

package version updated with several bug fix, refinement, new example#174
ArthurJiang merged 36 commits into
masterfrom
v0.1

Jinyu-W commented Oct 30, 2020 •

edited by ArthurJiang

Loading

Uh oh!

ArthurJiang commented Nov 4, 2020

Uh oh!

ArthurJiang commented Nov 4, 2020

Uh oh!

ArthurJiang commented Nov 4, 2020 •

edited

Loading

Uh oh!

Jinyu-W commented Nov 4, 2020

Uh oh!

Jinyu-W commented Nov 4, 2020

Uh oh!

Jinyu-W commented Nov 4, 2020

Uh oh!

codecov-io commented Nov 4, 2020 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Conversation

Jinyu-W commented Oct 30, 2020 • edited by ArthurJiang Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Linked issue(s)/Pull request(s)

Type of Change

Related Component

Has Been Tested

Needs Follow Up Actions

Checklist

Uh oh!

ArthurJiang commented Nov 4, 2020

Uh oh!

ArthurJiang commented Nov 4, 2020

Uh oh!

ArthurJiang commented Nov 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Jinyu-W commented Nov 4, 2020

Uh oh!

Jinyu-W commented Nov 4, 2020

Uh oh!

Jinyu-W commented Nov 4, 2020

Uh oh!

codecov-io commented Nov 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Jinyu-W commented Oct 30, 2020 •

edited by ArthurJiang

Loading

ArthurJiang commented Nov 4, 2020 •

edited

Loading

codecov-io commented Nov 4, 2020 •

edited

Loading