V0.2 fin alpha for review #170

zhawan · 2020-10-26T02:57:03Z

Description

alpha version of finance scenario

Linked issue(s)/Pull request(s)

Type of Change

Related Component

Simulation toolkit
RL toolkit
Distributed toolkit

Has Been Tested

OS:
- Windows
- Mac OS
- [✓] Linux
Python version:
- 3.6
- [✓] 3.7
Key information snapshot(s):

Needs Follow Up Actions

New release package
New docker image

Checklist

Add/update the related comments
Add/update the related tests
Add/update the related documentations
Update the dependent downstream modules usage

* prototype finance scenario * runnable finance demo * update for pylint * update for pylint * update for pylint

* prototype finance scenario * runnable finance demo * update for pylint * update for pylint * update for pylint * refactoring finance scenario part1 * part2 * part 3 * part 4 * remove unused code * remove not used file * update name * update for lint * update for lint

maro/simulator/scenarios/finance/README.md

maro/simulator/scenarios/finance/sub_engines/stock/stock_business_engine.py

Jinyu-W · 2020-10-26T03:37:34Z

maro/simulator/scenarios/finance/common/slippage.py

+        if order_action.number < 0:
+            order_direction = -1
+        actual_price = deal_price * (1 + self.__slippage_rate * order_direction / 2)
+        return actual_price


return the price delta instead of price in Slippage. Modify the code using slippage respectively.

still using price, can discuss

maro/simulator/scenarios/finance/account.py

maro/simulator/scenarios/finance/common.py

* prototype finance scenario * runnable finance demo * update for pylint * update for pylint * update for pylint * refactoring finance scenario part1 * part2 * part 3 * part 4 * remove unused code * remove not used file * update name * update for lint * update for lint * V0.2 rl toolkit refinement (#165) * refined rl abstractions * fixed formattin issues * checked out error-code related code from v0.2_pg * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * fixed a bug * renamed save_models to dump_models * 1. set default batch_norm_enabled to True; 2. used state_dict in dqn model saving * renamed dump_experience_store to dump_experience_pool * fixed a bug in the dump_experience_pool method * fixed some PR comments * fixed more PR comments * 1.fixed some PR comments; 2.added early_stopping_checker; 3.revised explorer class * fixed cim example according to rl toolkit changes * fixed some more PR comments * rewrote multi_process_launcher to eliminate the distributed section in config * 1. fixed a typo; 2. added logging before early stopping * fixed a bug * fixed a bug * fixed a bug * added early stopping feature to CIM exmaple * fixed a typo * fixed some issues with early stopping * changed early stopping metric func * fixed a bug * fixed a bug * added early stopping to dist mode cim * added experience collecting func * edited notebook according to changes in CIM example * fixed bugs in nb * fixed lint formatting issues * fixed a typo * fixed some PR comments * fixed more PR comments * revised docs * removed nb output * fixed a bug in simple_learner * fixed a typo in nb * fixed a bug * fixed a bug * fixed a bug * removed unused import * fixed a bug * 1. changed early stopping default config; 2. renamed param in early stopping checker and added typing * fixed some doc issues * added output to nb Co-authored-by: ysqyang <v-yangqi@microsoft.com> * finance business engine refine ver1 * day trading * remove not used files * refine cancel order * fix cancel order * add split trade support * fix split trade * remove not used file * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint Co-authored-by: ysqyang <ysqyang@gmail.com> Co-authored-by: ysqyang <v-yangqi@microsoft.com>

codecov-io · 2020-11-04T14:41:33Z

Codecov Report

Merging #170 (7087e95) into v0.2 (f7bd11a) will decrease coverage by 0.12%.
The diff coverage is 100.00%.

❗ Current head 7087e95 differs from pull request most recent head b08d806. Consider uploading reports for the commit b08d806 to get more accurate results

@@            Coverage Diff             @@
##             v0.2     #170      +/-   ##
==========================================
- Coverage   71.36%   71.23%   -0.13%     
==========================================
  Files          89       89              
  Lines        4190     4203      +13     
==========================================
+ Hits         2990     2994       +4     
- Misses       1200     1209       +9

Flag	Coverage Δ
unittests	`71.23% <100.00%> (-0.13%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
maro/simulator/core.py	`84.29% <ø> (ø)`
maro/simulator/abs_core.py	`78.46% <100.00%> (ø)`
maro/simulator/scenarios/abs_business_engine.py	`89.09% <100.00%> (ø)`
maro/simulator/scenarios/cim/business_engine.py	`85.76% <100.00%> (ø)`
...o/simulator/scenarios/citi_bike/business_engine.py	`75.10% <100.00%> (ø)`
...topologies/single_learner_multi_actor_sync_mode.py	`40.47% <0.00%> (-0.99%)`	⬇️
maro/rl/algorithms/dqn.py	`26.58% <0.00%> (+0.49%)`	⬆️
maro/rl/learner/simple_learner.py	`31.14% <0.00%> (+0.63%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f7bd11a...b08d806. Read the comment docs.

Jinyu-W · 2020-11-05T08:59:52Z

maro/simulator/scenarios/finance/README.md

+To simulate real markets, we implement the typical order types, cost of the trading.
+
+A typical trading process is:
+


invalid image path

* prototype finance scenario * runnable finance demo * update for pylint * update for pylint * update for pylint * refactoring finance scenario part1 * part2 * part 3 * part 4 * remove unused code * remove not used file * update name * update for lint * update for lint * finance business engine refine ver1 * day trading * remove not used files * refine cancel order * fix cancel order * add split trade support * fix split trade * remove not used file * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint * update for comments * update for lint * update for lint

Jinyu-W

Empty file? remaining TODO?

Jinyu-W · 2020-11-06T03:34:59Z

Empty file? remaining TODO?

misoperation, skip this

maro/simulator/scenarios/finance/business_engine.py

Jinyu-W · 2020-11-06T06:50:43Z

maro/simulator/scenarios/finance/business_engine.py

+            self._event_buffer.insert_event(evt)
+
+    def post_step(self, tick: int):
+        if (not self._conf["trade_constraint"]["allow_day_trade"]) and self.is_market_closed():


Add detail comment for this part. Why we add holding_quantity here, and how we make sure the operation is valid.

maro/simulator/scenarios/finance/business_engine.py

Jinyu-W · 2020-11-06T07:46:47Z

maro/simulator/scenarios/finance/business_engine.py

+        if direction == OrderDirection.buy:
+            money_needed = executing_price * executing_volume + executing_commission + executing_tax
+            ret = ret - math.ceil(
+                (money_needed - remaining_money) / (executing_price * self._conf["trade_constraint"]["min_buy_unit"])


should we initialize
self._min_buy_unit = self._conf["trade_constraint"]["min_buy_unit"]
in self._init()?

or using other meaning name

maro/simulator/scenarios/finance/business_engine.py

Jinyu-W · 2020-11-06T09:54:00Z

maro/simulator/scenarios/finance/business_engine.py

+
+        return DocableDict(
+            metrics_desc,
+            operation_number=len(self._finished_action)


We can have more discussion about the metrics with Arthur together.

maro/simulator/scenarios/finance/business_engine.py

ArthurJiang · 2020-11-09T06:25:55Z

Fulfill the pr description, such as data source, feature list, etc.

ArthurJiang · 2020-11-09T06:31:54Z

why we need the 0000X.bin, we never host data, if it was test purpose, just keep one, and mv it to maro/tests/data/finance or maro/tests/data/stock

maro/simulator/scenarios/finance/business_engine.py

maro/simulator/scenarios/finance/common/commission.py

Jinyu-W · 2020-11-12T03:24:48Z

maro/simulator/scenarios/finance/common/commission.py

+
+class CommissionType(Enum):
+    no_commission = 0
+    stamp_tax_commission = 1


since there are both tax and commission in TradeResult, they are 2 types of cost? But here the stamp_tax_commission belongs to commission ?

Jinyu-W · 2020-11-12T06:42:47Z

maro/simulator/scenarios/finance/common/slippage.py

+
+class SlippageType(Enum):
+    # TODO: zhanyu add correct slippage
+    no_slippage = 0


if currently only by_money is available, remove the others from the Type.

Or, add them but throw NotImplementationError

maro/simulator/scenarios/finance/frame_builder.py

maro/simulator/scenarios/finance/stock/stock.py

Jinyu-W · 2020-11-12T07:24:37Z

maro/simulator/scenarios/finance/common/commission.py

+    def __init__(self, min_fee):
+        self.min_fee = min_fee
+
+    def execute(self, direction: OrderDirection, actual_price: float, actual_volume: int):


maro/simulator/scenarios/finance/common/commission.py

maro/simulator/scenarios/finance/stock/stock.py

* prototype finance scenario * runnable finance demo * update for pylint * update for pylint * update for pylint * refactoring finance scenario part1 * part2 * part 3 * part 4 * remove unused code * remove not used file * update name * update for lint * update for lint * finance business engine refine ver1 * day trading * remove not used files * refine cancel order * fix cancel order * add split trade support * fix split trade * remove not used file * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint * update for comments * update for lint * update for lint * update for comments * update for comments part 1 * update for comments * update for comments part2 * update for lint * add help function for convert start tick * update for lint * remove not used import

examples/hello_world/finance/simple_buy_sell.py

examples/hello_world/finance/simple_strategy.py

maro/simulator/scenarios/finance/account.py

maro/simulator/scenarios/finance/business_engine.py

examples/hello_world/finance/simple_buy_sell.py

maro/simulator/scenarios/finance/business_engine.py

* prototype finance scenario * runnable finance demo * update for pylint * update for pylint * update for pylint * refactoring finance scenario part1 * part2 * part 3 * part 4 * remove unused code * remove not used file * update name * update for lint * update for lint * finance business engine refine ver1 * day trading * remove not used files * refine cancel order * fix cancel order * add split trade support * fix split trade * remove not used file * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint * update for lint * update for comments * update for lint * update for lint * update for comments * update for comments part 1 * update for comments * fixed a bug in learner's test() (#193) Co-authored-by: ysqyang <v-yangqi@microsoft.com> * V0.2 double dqn (#188) * added dueling action value model * renamed params in dueling_action_value_model * renamed shared_features to features * replaced IdentityLayers with nn.Identity * 1. added skip connection option in FC_net; 2. generalized learning model * added skip_connection option in config * removed type casting in fc_net * fixed lint formatting issues * refined docstring * mv dueling_actiovalue_model and fixed some bugs * added multi-head functionality to LearningModel * refined learning model docstring * added head_key param in learningModel forward * added double DQN and dueling features to DQN * fixed a bug * added DuelingQModelHead enum * fixed a bug * removed unwanted file * fixed PR comments * added top layer logic and is_top option in fc_net * fixed a bug * fixed a bug * reverted some changes in learning model * reverted some changes in learning model * added members to learning model to fix the mode issue * fixed a bug * fixed mode setting issue in learning model * fixed PR comments * revised cim example according to DQN changes * renamed eval_model to q_value_model in cim example * more fixes * fixed a bug * fixed a bug * added doc per PR comments * removed learner.exit() in single_process_launcher * removed learner.exit() in single_process_launcher * fixed PR comments * fixed rl/__init__ * fixed issues in example * fixed a bug * fixed a bug * fixed lint formatting issues * double DQN feature * fixed a bug * fixed a bug * fixed PR comments * fixed lint issue * 1. fixed PR comments related to load/dump; 2. removed abstract load/dump methods from AbsAlgorithm * added load_models in simple_learner * minor docstring edits * minor docstring edits * set is_double to true in DQN config Co-authored-by: ysqyang <v-yangqi@microsoft.com> Co-authored-by: Arthur Jiang <ArthurSJiang@gmail.com> * update for comments part2 * V0.2 feature predefined image (#183) * feat: support predefined image provision * style: fix linting errors * style: fix linting errors * style: fix linting errors * style: fix linting errors * fix: error scripts invocation after using relative import * fix: missing init.py * fixed a bug in learner's test() * feat: add distributed_config for dqn example * test: update test for grass * test: update test for k8s * feat: add promptings for steps * fix: change relative imports to absolute imports Co-authored-by: ysqyang <v-yangqi@microsoft.com> Co-authored-by: Arthur Jiang <ArthurSJiang@gmail.com> * update for lint * add help function for convert start tick * update for lint * remove not used import * update for review comments 11.17 * refine code for review 11.18 * V0.2 feature proxy rejoin (#158) * update dist decorator * replace proxy.get_peers by proxy.peers * update proxy rejoin (draft, not runable for proxy rejoin) * fix bugs in proxy * add message cache, and redesign rejoin parameter * feat: add checkpoint with test * update proxy.rejoin * fixed rejoin bug, rename func * add test example(temp) * feat: add FaultToleranceAgent, refine other MasterAgents and NodeAgents. * capital env vari name * rm json.dumps; change retries to 10; temp add warning level for rejoin * fix: unable to load FaultToleranceAgent, missing params * fix: delete mapping in StopJob if FaultTolerance is activated, add exception handler for FaultToleranceAgent * feat: add node_id to node_details * fix: add a new dependency for tests * style: meet linting requirements * style: remaining linting problems * lint fixed; rm temp test folder. * fixed lint f-string without placeholder * fix: add a flag for "remove_container", refine restart logic and Redis keys naming * proxy rejoin update. * variable rename. * fixed lint issues * fixed lint issues * add exit code for different error * feat: add special errors handler * add max rejoin times * remove unused import * add rejoin UT; resolve rejoin comments * lint fixed * fixed UT import problem * rm MessageCache in proxy * fix: refine key naming * update proxy rejoin; add topic for broadcast * feat: support predefined image provision * update UT for communication * add docstring for rejoin * fixed isort and zmq driver import * fixed isort and UT test * fix isort issue * proxy rejoin update (comments v2) * fixed isort error * style: fix linting errors * style: fix linting errors * style: fix linting errors * style: fix linting errors * feat: add exists method for checkpoint * fix: error scripts invocation after using relative import * fix: missing init.py * fixed a bug in learner's test() * add driver close and socket SUB disconnect for rejoin * feat: add distributed_config for dqn example * test: update test for grass * test: update test for k8s * feat: add promptings for steps * fix: change relative imports to absolute imports * fixed comments and update logger level * mv driver in proxy.__init__ for issue temp fixed. * Update docstring and comments * style: fix code reviews problems * fix code format Co-authored-by: Lyuchun Huang <romic.kid@gmail.com> Co-authored-by: ysqyang <v-yangqi@microsoft.com> * update for review comments 11.18 part 2 * update for review comments 11.18 part3 * remove unfinished code * update for lint Co-authored-by: ysqyang <ysqyang@gmail.com> Co-authored-by: ysqyang <v-yangqi@microsoft.com> Co-authored-by: Arthur Jiang <ArthurSJiang@gmail.com> Co-authored-by: Romic Huang <romic.kid@gmail.com> Co-authored-by: kaiqli <59279714+kaiqli@users.noreply.github.com>

Jinyu-W · 2022-05-13T08:52:35Z

Reopen or create a new one when it is ready

zhawan added 2 commits October 19, 2020 16:01

V0.2 (#161)

07c3794

* prototype finance scenario * runnable finance demo * update for pylint * update for pylint * update for pylint

Jinyu-W requested changes Oct 26, 2020

View reviewed changes

Jinyu-W assigned zhawan Oct 29, 2020

Jinyu-W added in progress Not finished items/bugs. v0.2 labels Oct 29, 2020

Jinyu Wang and others added 4 commits October 29, 2020 07:41

Merge branch 'v0.2' into v0.2_fin_alpha

a0cde6d

merge with v0.2

b6b2d0e

update according to lint isort

80e1199

Jinyu Wang added 4 commits November 4, 2020 14:51

update according to lint isort

65aaf92

update according to isort lint

61c2bf6

update according to isort lint

5066452

merge with v0.2

1795c47

Jinyu-W reviewed Nov 5, 2020

View reviewed changes

Jinyu Wang added 3 commits November 5, 2020 09:37

update according to isort

74c960f

merge with v0.2

3fef971

Merge branch 'v0.2' into v0.2_fin_alpha

d58c359

Jinyu-W added the finance finance scenario. label Nov 6, 2020

Jinyu-W reviewed Nov 6, 2020

View reviewed changes

maro/simulator/scenarios/finance/business_engine.py Outdated Show resolved Hide resolved

Jinyu-W requested changes Nov 6, 2020

View reviewed changes

merge with v0.2

64ac719

Jinyu-W requested changes Nov 6, 2020

View reviewed changes

ArthurJiang reviewed Nov 12, 2020

View reviewed changes