How to change the agent type?

[lamhk]

Hi, I tried to build "-b BUILD" using DQN instead of PPO inside run.py line-10 / agent = "PPO". However, when I try to run the run.py, it still create directory for PPO_n. Where I should change it to switch the agent effectively. Thanks.

[lamhk]

Hi, i tried to change (1) session/local.py with "dqn"; (2) run.py agent="DQN" and (3) config/agent.json with "type": "dqn_agent" and saw it successfully create save/DQN_0 directory. However, I received the following error...

Traceback (most recent call last):
File "run.py", line 56, in
session.loadSession()
File "/home/lamhk/TradzQAI-master/core/session/local.py", line 47, in loadSession
self.initAgent()
File "/home/lamhk/TradzQAI-master/core/session/local.py", line 58, in initAgent
self.agent = self.agent(env=self.env, device=self.device)._get()
File "/home/lamhk/TradzQAI-master/agents/DQN.py", line 6, in init
Agent.init(self, env=env, device=device)
File "/home/lamhk/TradzQAI-master/agents/agent.py", line 23, in init
device=device
File "/home/lamhk/.local/lib/python3.6/site-packages/tensorforce/agents/agent.py", line 283, in from_spec
kwargs=kwargs
File "/home/lamhk/.local/lib/python3.6/site-packages/tensorforce/util.py", line 192, in get_object
return obj(*args, **kwargs)
TypeError: init() got an unexpected keyword argument 'gae_lambda'

[kkuette]

You just have to delete your current TradzQAI/config/ directory and build a new one with this command py run.py -b DQN.

[lamhk]

Hi, if I deleted the TradzQAI/config, i received the following error (after running py run.py -b DQN) . If the config directory exists, there is no error but the above 'gae_lamdba' error happens again. Any idea? Thanks.

Traceback (most recent call last):
File "run.py", line 46, in
session = Session(mode=args.mode, config=args.config, agent=args.build)
File "/home/lamhk/TradzQAI-master/core/session/local.py", line 15, in init
self.env = Local_env(mode=mode, gui=gui, contract_type=contract_type, config=config, agent=agent)
File "/home/lamhk/TradzQAI-master/core/environnement/local_env.py", line 98, in init
self.close()
File "/home/lamhk/TradzQAI-master/core/environnement/base/base_env.py", line 71, in close
self.logger.stop()
AttributeError: 'Local_env' object has no attribute 'logger'

[kkuette]

I've fixed the problem, it was due to a hard coded argument. Commit done !

[lamhk]

Hi, Thanks for the updated code. I tried using DQN agent as mentioned above with 100 episodes. After the first episode, the rest of the episodes doesn't have result. I checked the eval report also shows zero result.

2018:10:03 13:49:06 000000 Starting episode : 1
2018:10:03 13:49:44 000001 ######################################################
2018:10:03 13:49:44 000002 Total reward : -238.704
2018:10:03 13:49:44 000003 Average daily reward : -10.850
2018:10:03 13:49:44 000004 Total profit : -240.25
2018:10:03 13:49:44 000005 Total trade : 465
2018:10:03 13:49:44 000006 Sharp ratio : -1.723
2018:10:03 13:49:44 000007 Mean return : -0.013
2018:10:03 13:49:44 000008 Max Drawdown : -0.019
2018:10:03 13:49:44 000009 Max return : 0.002
2018:10:03 13:49:44 000010 Percent return : -0.012
2018:10:03 13:49:44 000011 Trade W/L : 0.495
2018:10:03 13:49:44 000012 Step : 18353
2018:10:03 13:49:44 000013 ######################################################
2018:10:03 13:49:44 000014 Starting episode : 2
2018:10:03 13:50:17 000015 ######################################################
2018:10:03 13:50:17 000016 Total reward : 0.0
2018:10:03 13:50:17 000017 Average daily reward : 0.000
2018:10:03 13:50:17 000018 Total profit : 0
2018:10:03 13:50:17 000019 Total trade : 0
2018:10:03 13:50:17 000020 Sharp ratio : 0.000
2018:10:03 13:50:17 000021 Mean return : 0.000
2018:10:03 13:50:17 000022 Max Drawdown : 0.000
2018:10:03 13:50:17 000023 Max return : 0.000
2018:10:03 13:50:17 000024 Percent return : 0.000
2018:10:03 13:50:17 000025 Trade W/L : 0.000
2018:10:03 13:50:17 000026 Step : 18353

[kkuette]

Defaults values aren't the best values for your agent, you should run some test to find out the best values.
If it shows results like yours it's because your model does not learn well.