Skip to content

V0.2_refactored_distributed_framework #206

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ysqyang merged 555 commits into from
Mar 18, 2021
Merged

V0.2_refactored_distributed_framework #206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
555 commits
Select commit Hold shift + click to select a range
278a86b
conflict fix
Dec 16, 2020
3993f25
conflict fix
Dec 16, 2020
a1b2e0f
Merge remote-tracking branch 'origin/v0.2' into v0.2_explorer
Dec 16, 2020
1b1316d
Merge branch 'v0.2_explorer' into v0.2_embedded_optim
Dec 16, 2020
61884e1
Merge branch 'v0.2_explorer' into v0.2_seed
Dec 16, 2020
8deaecc
Merge branch 'v0.2_embedded_optim' into v0.2_seed
Dec 16, 2020
48af852
added some more logs for dist RL
Dec 16, 2020
8fbbbf6
bug fix
Dec 16, 2020
8aff613
fixed a typo
Dec 16, 2020
3e6952c
bug fix
Dec 16, 2020
6e6d418
refined logs
Dec 16, 2020
5258fa1
set session_id to None for exit message
Dec 16, 2020
580d0ec
add setup/clear/template for maro process
Dec 17, 2020
ab1a4c2
changed to internal logger for actor and learner
Dec 17, 2020
82b27f0
Merge remote-tracking branch 'origin/v0.2_local_host_process' into v0…
Dec 17, 2020
265b7cb
removed redundant component name from internal logs
Dec 17, 2020
e8ba54b
fix process stop
Dec 17, 2020
4e4ecde
add logger and rename parameters
Dec 18, 2020
06921b7
add logger for setup/clear
Dec 18, 2020
c89e68f
fixed close not exist pid when given pid list.
Dec 18, 2020
6682a81
Merge remote-tracking branch 'origin/v0.2' into v0.2_local_host_process
Dec 21, 2020
e0f59aa
Fixed comments and rename setup/clear with create/delete
Dec 21, 2020
aa3b89c
fixed typos
Dec 21, 2020
e8ba161
update ProcessInternalError
Dec 22, 2020
57dbe15
Merge remote-tracking branch 'origin/v0.2_local_host_process' into v0…
Dec 22, 2020
fd67efd
removed explorer abstraction from agent
Dec 22, 2020
5b5d7bc
Merge remote-tracking branch 'origin/v0.2' into v0.2_explorer
Dec 22, 2020
5f7f73f
fixed conflicts
Dec 22, 2020
b29fd13
added DEVICE env variable as first choice for torch device
Dec 23, 2020
7cb9388
refined dqn example
Dec 23, 2020
f8dd9bd
Merge remote-tracking branch 'origin/v0.2' into v0.2_explorer
Dec 23, 2020
ec3e3f4
Merge branch 'v0.2_explorer' into v0.2_embedded_optim
Dec 23, 2020
236c9fc
fixed lint issues
Dec 23, 2020
ecef225
Merge branch 'v0.2_explorer' into v0.2_embedded_optim
Dec 23, 2020
0209a98
removed unwanted import in cim example
Dec 23, 2020
40ffa26
fixed conflicts
Dec 23, 2020
804fd08
updated cim-dqn notebook
Dec 23, 2020
9dd71d6
simplified scheduler
Dec 24, 2020
3addbff
Merge remote-tracking branch 'origin/v0.2' into v0.2_explorer
Dec 24, 2020
87beb2b
fixed conflicts
Dec 24, 2020
bc7b841
edited notebook according to merged scheduler changes
Dec 24, 2020
8ad6378
refined dimension check for learning module manager and removed num_a…
Dec 24, 2020
d4fe0db
bug fix for cim example
Dec 24, 2020
9fe7590
added notebook output
Dec 24, 2020
ccf14c0
removed early stopping from CIM dqn example
Dec 24, 2020
af32156
fixed conflicts
Dec 24, 2020
7de7c25
fixed conflicts
Dec 24, 2020
382108e
removed early stopping from cim example config
Dec 24, 2020
13f7f04
Merge branch 'v0.2_explorer' into v0.2_embedded_optim
Dec 24, 2020
e9d6558
Merge branch 'v0.2_embedded_optim' into v0.2_seed
Dec 24, 2020
20200a2
updated notebook
Dec 24, 2020
7e7338a
1. removed external loggers from cim example; 2. fixed batch inferenc…
Dec 25, 2020
da9cac7
removed actor_trainer mode and refactored
Dec 25, 2020
407fb9a
moved decorator logic inside algorithms
Dec 25, 2020
be9cbeb
Merge remote-tracking branch 'origin/v0.2' into v0.2_explorer
Dec 25, 2020
61e869e
renamed early_stopping_callback to early_stopping_checker
Dec 25, 2020
d1272eb
Merge remote-tracking branch 'origin/v0.2' into v0.2_embedded_optim
Dec 25, 2020
b55c5d8
small refinement
Dec 25, 2020
e51811d
fixed conflicts
Dec 26, 2020
c8a522c
fixed conflicts
Jan 5, 2021
4a40f58
fixed conflicts
Jan 5, 2021
a7c3b7d
fixed typos
Jan 5, 2021
83a9786
removed stale imports
Jan 5, 2021
2b25860
fixed stale naming
Jan 5, 2021
d7149ba
Merge remote-tracking branch 'origin/master' into v0.2_seed
Jan 6, 2021
3071ba8
Merge branch 'master' into v0.2_seed
Jan 6, 2021
7dcf2f5
removed dist_topologies folder
Jan 6, 2021
5e7d0ae
refined session id logic
Jan 6, 2021
52b5431
bug fix
Jan 8, 2021
d3144a8
refactored
Jan 12, 2021
1c75e03
distributed RL refinement
Jan 12, 2021
fc78417
Merge remote-tracking branch 'origin/master' into v0.2_seed
Jan 12, 2021
3dc54c4
refined
Jan 14, 2021
8264198
small bug fix
Jan 14, 2021
0f57927
fixed lint issues
Jan 14, 2021
e955eb7
fixed merge conflicts
Jan 14, 2021
f1fd5bf
fixed lint issues
Jan 14, 2021
1d9c7c2
removed unwanted file
Jan 14, 2021
067e8b1
fixed a typo
Jan 15, 2021
254ce6f
Merge remote-tracking branch 'origin/v0.2' into v0.2_seed
Jan 19, 2021
b50c1a9
Merge remote-tracking branch 'origin/v0.2' into v0.2_seed
Jan 20, 2021
ecdb17f
gnn refactoring in progress
Jan 22, 2021
7e84c42
merged algorithm with agent
Jan 22, 2021
57d9cfe
bug fixes
Jan 22, 2021
431d55c
fix
Jan 22, 2021
b6075af
Merge branch 'v0.2_merge_algorithm_into_agent' of github.com:microsof…
Jan 22, 2021
296cccf
bug fixes
Jan 22, 2021
a87b867
fixed lint issues and renamed models to model
Jan 22, 2021
485bea7
fixed merge conflicts
Jan 22, 2021
b526ae6
removed unwanted files
Jan 22, 2021
6fda836
fixed merge conflicts
Jan 22, 2021
fa200e9
removed exp pool type spec in AbsAgent
Jan 23, 2021
7bad295
fixed lint issues
Jan 23, 2021
9aa5142
Merge branch 'v0.2_merge_algorithm_into_agent' into v0.2_seed
Jan 23, 2021
5fc4c26
changed to a single gnn agent
Jan 23, 2021
bdb86de
dqn exp pool bug fix
Jan 23, 2021
0e5f566
Merge branch 'v0.2_merge_algorithm_into_agent' of github.com:microsof…
Jan 23, 2021
8174a22
minor issues
Jan 23, 2021
b56b1d5
removed GNNAgentManager
Jan 23, 2021
e27e046
Merge remote-tracking branch 'origin/v0.2_merge_algorithm_into_agent'…
Jan 23, 2021
b03f9b2
Merge remote-tracking branch 'origin/v0.2' into v0.2_merge_algorithm_…
Jan 25, 2021
405e844
updated notebooks and examples according to rl toolkit changes
Jan 26, 2021
a634d44
updated images
Jan 26, 2021
d063945
moved exp pool init inside DQN
Jan 26, 2021
9912828
renamed column_based_store to simple_store
Jan 26, 2021
b92cd1e
mroe gnn refactoring
Jan 26, 2021
d7ecb8a
fixed merge conflicts
Jan 26, 2021
4e8b5ac
fixed lint issues
Jan 26, 2021
0371a79
Merge remote-tracking branch 'origin/v0.2_merge_algorithm_into_agent'…
Jan 26, 2021
27ab553
fixed lint issues
Jan 26, 2021
639e419
lint issue fix
Jan 26, 2021
962534a
lint issue fix
Jan 26, 2021
4dca910
fixed bugs in test_store
Jan 26, 2021
70d0db0
revised ddpg according to rl abstraction changes
Jan 26, 2021
ebb5ffd
typo fix
Jan 26, 2021
97312b6
minor edits
Jan 26, 2021
d0a5b26
fixed merge conflicts
Jan 26, 2021
d743b4d
Merge remote-tracking branch 'origin/v0.2_merge_algorithm_into_agent'…
Jan 26, 2021
30e45bd
lint issue fix
Jan 26, 2021
13be114
finished single process gnn
Jan 26, 2021
cdf5edc
fixed bugs
Jan 27, 2021
da87dd9
1. removed state_shaper, action_shaper and exp_shaper abstractions; 2…
Jan 27, 2021
2e27782
updated notebook
Jan 27, 2021
7216b14
fixed merge conflicts
Jan 27, 2021
e33ddd2
removed simple agent manager
Jan 27, 2021
c0e1385
fixed lint issues
Jan 27, 2021
73b9534
fixed lint issues
Jan 27, 2021
649b38b
bug fix
Jan 27, 2021
71205c3
fixed merge conflicts
Jan 27, 2021
ff0c793
bug fixes
Jan 28, 2021
e90aaaa
refined LearningModel
Jan 28, 2021
52788f8
Merge branch 'v0.2_merge_algorithm_into_agent' into v0.2_seed
Jan 28, 2021
b88fce7
modified gnn example based on latest rl toolkit changes
Jan 28, 2021
c215306
updated cim example doc
Jan 28, 2021
932c99d
Merge branch 'v0.2_merge_algorithm_into_agent' into v0.2_seed
Jan 28, 2021
132e97e
lint issue fix
Jan 28, 2021
066f19b
Merge branch 'v0.2_merge_algorithm_into_agent' into v0.2_seed
Jan 28, 2021
bf12ec6
small refinements
Jan 28, 2021
888d319
Merge branch 'v0.2_merge_algorithm_into_agent' into v0.2_seed
Jan 28, 2021
555ae78
refactored GNN example
Jan 29, 2021
b35b1eb
replaced ActionInfo with torch Categorical's log_prob for policy_opti…
Jan 29, 2021
c392c09
refactored gnn example and added single-process script
Jan 29, 2021
53e17c7
removed obsolete files from gnn
Jan 29, 2021
c481de1
lint issue fix
Jan 29, 2021
201eb61
Merge remote-tracking branch 'origin/v0.2' into v0.2_merge_algorithm_…
Jan 29, 2021
79ce1be
Merge remote-tracking branch 'origin/v0.2' into v0.2_gnn_refactoring
Jan 29, 2021
f3f2704
fixed merge conflicts
Jan 29, 2021
9f4ca9a
fixed merge conflicts
Jan 29, 2021
ca964ac
formatting
Jan 29, 2021
c3046bf
Merge branch 'v0.2_merge_algorithm_into_agent' into v0.2_gnn_refactoring
Jan 29, 2021
4c23386
Merge branch 'v0.2_gnn_refactoring' into v0.2_seed
Jan 29, 2021
411309b
checked out gnn files from origin/v0.2
Feb 1, 2021
612635a
fixed merge conflicts
Feb 1, 2021
56ddcc1
Merge remote-tracking branch 'origin/v0.2' into v0.2_seed
Feb 1, 2021
22a252c
refactored distributed rl toolkit
Feb 3, 2021
a818aed
finished distributed rl refactoring and updated dqn example and notebook
Feb 4, 2021
884535d
merged request_rollout with collect
Feb 4, 2021
37e91eb
some refinement
Feb 5, 2021
a16bb69
refactored examples
Feb 5, 2021
fd7d195
distributed rl revamping complete
Feb 9, 2021
4c4b0e4
Merge remote-tracking branch 'origin/v0.2' into v0.2_seed
Feb 9, 2021
cb838a2
bug and formatting fixes
Feb 9, 2021
4604ae7
bug fixes
Feb 9, 2021
b5894f2
hid proxy instantiation inside dist components
Feb 9, 2021
36c9099
Merge remote-tracking branch 'origin/v0.2' into v0.2_seed
Feb 9, 2021
bdc8ea0
small refinement
Feb 9, 2021
bcebab0
refined distributed RL and updated docs
Feb 10, 2021
e0b8a17
updated docs and notebook
Feb 13, 2021
131229a
rm unwanted imports
Feb 13, 2021
62e529c
added missing files
Feb 13, 2021
f21cd76
rm unwanted files
Feb 13, 2021
41ae13f
lint issue fix
Feb 13, 2021
04e14ec
bug fix
Feb 14, 2021
0ea94c1
example doc update
Feb 15, 2021
45e59b9
rm agent_manager.svg
Feb 15, 2021
cbcf48b
updated images
Feb 15, 2021
0959ac5
updated image file name in doc
Feb 15, 2021
defaa56
revamped cim example code structure
Feb 19, 2021
8dc5bf0
Merge remote-tracking branch 'origin/v0.2' into v0.2_seed
Feb 19, 2021
80155eb
added missing file
Feb 19, 2021
2706f26
restored default training config for dqn and ac-gnn
Feb 19, 2021
5aad368
added default loss function for actor-critic
Feb 22, 2021
62cb3ef
fixed merge conflicts
Feb 22, 2021
e102711
rm unwanted import
Feb 22, 2021
29b1be0
updated README for cim/ac
Feb 23, 2021
a05db63
removed log_p param for PolicyGradient train()
Feb 24, 2021
3fdc89b
added READMEs for CIM
Feb 26, 2021
059af81
renamed ac-gnn to ac_gnn
Feb 26, 2021
bae7e69
updated README for CIM and added set_seeds to multi-process dqn
Feb 26, 2021
27587fc
init
chaosddp Mar 3, 2021
0291b7d
remove unit, make it same as logic
chaosddp Mar 4, 2021
0e7fdec
init by sku, world sku
chaosddp Mar 4, 2021
0b723df
init by sku, world sku
chaosddp Mar 4, 2021
74c47db
remove debug code
chaosddp Mar 4, 2021
b0b7230
correct snapshot number issue
chaosddp Mar 4, 2021
8c3cfc8
rename logic to unit, make it meaningful
chaosddp Mar 4, 2021
2a9ef02
add facility base
chaosddp Mar 4, 2021
5b64ec7
refine naming
chaosddp Mar 4, 2021
fcd4ffd
refine the code, more comment to make it easy to read
chaosddp Mar 4, 2021
4cfece0
add supplier facility, logic not tested yet
chaosddp Mar 4, 2021
51f9349
fix bug in facility initialize, add consumerunit not completed
chaosddp Mar 4, 2021
9fd1b15
refactoring the facilities in world config
chaosddp Mar 5, 2021
dd62940
add consumer for warehouse facility
chaosddp Mar 5, 2021
f60f680
add upstream topology, and save it state
chaosddp Mar 5, 2021
2aeb783
add mapping from id to data model index
chaosddp Mar 5, 2021
93b3bb5
logic without reward of consumer
chaosddp Mar 5, 2021
8ef747c
bug fix
chaosddp Mar 5, 2021
0a44175
seller unit
chaosddp Mar 5, 2021
dc9c2e5
use tcod for path finding
chaosddp Mar 5, 2021
8847e8f
retailer facility
chaosddp Mar 5, 2021
47a3aa7
bug fix, show seller demands in example
chaosddp Mar 5, 2021
9d2289a
add a interactive and renderable env wrapper to later debugging
chaosddp Mar 6, 2021
266f10d
move font to subfolder with lisence to make it more clearly
chaosddp Mar 6, 2021
af2ac89
add more details for node mapping
chaosddp Mar 6, 2021
e6ef16a
dispatch action by unit id
chaosddp Mar 6, 2021
0ce7abb
merge the frame changes to support data model inherit
chaosddp Mar 6, 2021
17996b0
add action for consumer, so that we can push the requirement
chaosddp Mar 6, 2021
92e8240
add unit id and facility in state for unit, add storage id for manufa…
chaosddp Mar 6, 2021
27fa332
show manufacture related debug info step by step
chaosddp Mar 6, 2021
617e9c4
add bom info for debug
chaosddp Mar 6, 2021
d9719d5
add x,y to facility, bug fix
chaosddp Mar 6, 2021
3f375b9
fix bugs in transport and distribution unit, correct the path finding…
chaosddp Mar 6, 2021
84807f2
show vehicle movement in screen
chaosddp Mar 6, 2021
2aa2637
remove completed todo
chaosddp Mar 6, 2021
711b475
fix vehicle location issue, make all units and data model class from …
chaosddp Mar 7, 2021
83bd035
show more states
chaosddp Mar 7, 2021
f82022a
fix slot number bug for dynamic backend
chaosddp Mar 7, 2021
e8193c6
rename suppliers to manufactures
chaosddp Mar 8, 2021
e3ad3e5
add missing file
chaosddp Mar 8, 2021
19ed33e
remove code config, use yml instead
chaosddp Mar 8, 2021
ee11adf
add 2 different step modes
chaosddp Mar 8, 2021
99e0b67
update changes
chaosddp Mar 8, 2021
af2cfe9
rename manufacture
chaosddp Mar 9, 2021
f4ec67a
add action for manufacture unit
chaosddp Mar 9, 2021
8edd14c
more attribute for states
chaosddp Mar 9, 2021
24a830a
add balance sheet
chaosddp Mar 9, 2021
d3acb67
rename attribute to unify the feature name
chaosddp Mar 10, 2021
db81bc9
Merge branch 'supplychain_prototype2' of github.com:chaosddp/maro int…
Mar 10, 2021
09fb567
reverted experimental changes in dqn learner
Mar 12, 2021
3f84097
updated notebook
Mar 15, 2021
abfa88e
rm supply chain code
Mar 15, 2021
e5cf207
lint issue fix
Mar 15, 2021
d3f30d4
lint issue fix
Mar 15, 2021
0c7c550
added missing file
Mar 15, 2021
71effb2
added general rollout workflow and trajectory class
Mar 16, 2021
582c82a
refactored
Mar 17, 2021
2b8b9a0
Merge remote-tracking branch 'origin/v0.2' into v0.2_seed
Mar 17, 2021
35f3802
more refactoring
Mar 18, 2021
8088299
checked out backend from v0.2
Mar 18, 2021
f658a12
checked out setup.py from v0.2
Mar 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
412 changes: 135 additions & 277 deletions docs/source/examples/multi_agent_dqn_cim.rst
View file
Open in desktop

Large diffs are not rendered by default.

3 changes: 0 additions & 3 deletions docs/source/images/rl/agent_manager.svg
View file
Open in desktop

This file was deleted.

3 changes: 3 additions & 0 deletions docs/source/images/rl/learner_actor.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 0 additions & 3 deletions docs/source/images/rl/overview.svg
View file
Open in desktop

This file was deleted.

Loading