MineRLNavigate-v0 Trajectory continues for 164 frames after termination

[ryanrudes]

There is a trajectory in MineRLNavigate-v0 where the user receives a cumulative reward of 16400, although the maximum cumulative reward in this env should be 100. Upon touching the diamond, the user receives the sparse reward signal 164 times, but the episode should terminate immediately upon making contact with the target block.

[ryanrudes]

Nevermind, there was a bug in my data processor. While the user does only receive the reward once, the episode continues on for another 164 frames despite what should have been episode termination.

[shwang]

Could you post the name of this demonstration? (Of the form adjective-plant-animal-#-start_timestamp-end_timestamp)

[ryanrudes]

v3_absolute_grape_changeling-47_826-1734

[ryanrudes]

There were also quite a few problems with the MineRLTreechop-v0 data that I think should be pointed out.

In the trajectory labeled "v3_colorless_mung_bean_dragon-19_716-3781", the user accumulates a total reward of 68, although the episode should terminate once it reaches 64.

There are over 20 other trajectories in which the user receives a cumulative reward of 65, one beyond the theoretical maximum. It's needless to mention the identifiers of all of these.

[shwang]

Thanks for pointing this out -- I'll keep an eye out for these problems when I process and upload the next version of the dataset (I expect that this will happen within the next two weeks).

Any idea why we might be seeing this sort of behavior (both the reward=65 and the "navigation not ending for 100 frames")? @brandonhoughton

In the case of the "navigate not ending for 100 frames", I'm thinking this is probably just corrupted demonstration data? The quickest solution here is probably to mark that particular trajectory name as corrupted.

[brandonhoughton]

It could be that the human placed a block and got reward a second time for mining a log

[shwang]

It could be that the human placed a block and got reward a second time for mining a log

That makes sense, since what we are actually checking to create the episode termination condition is "there are 64 logs in the inventory"

[shwang]

I (and likely Cody) don't have the bandwidth to understand the relevant parts of the Dataset Pipeline and modify them to fix this issue right now.