environement.step() reset action

[luvf]

from runner.log : lines 81-84

action = self.agent.act(observation)
observation, reward_aslist, done, info = self.environment.step(action, do_sum=False)

when action is object Action (and not array)
after environement.step(..)
the action is reset to the Do_Nothing action

Note that the action is applied.

but it is a bit inconsistent
and when,t action is an np.array this doesn't happen.

[MarvinLer]

What is the problem here? The reset of the Action into do-nothing ? Maybe the simulator changed the action because:

• there are illegal activations of on-cooldown switches
• there are more activated elements than tolerated, in that case the action is indeed converted into a do-nothing

As for the difference with np array, this is because you cannot change a np array by reference in python, but an object passed as reference can be modified within a function.

Do you mean in both case the action object of the agent should be replaced in-place or not replaced at all ?

[luvf]

as far as I know the action applied is legal : for (GreedySearch, and an expert agent)
happens at each timestep (even the first one (no cooldown))

I think the object shouldn't be modified.

[MarvinLer]

Okay I'll look into it in a couple of hours and make it such that Action type objects are not modified with environment.step

…

On Mon, May 6, 2019 at 4:10 PM Luca Veyrin-Forrer ***@***.***> wrote: as far as I know the action applied is legal : for (GreedySearch, and an expert agent) happens at each timestep (even the first one (no cooldown)) I think the object shouldn't be modified. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#39 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADMF4JP2TGM6H2IZWYM55ZDPUA34BANCNFSM4HLAGSJA> .

[MarvinLer]

This should be fixed in 536100b, can you confirm ?