rearrangement benchmark (2022) openness

[Gabesarch]

I am opening this to discuss a few items related to the most recent rearrangement benchmark (2022):

1. We noticed that when executing the “open” command in the unshuffle phase on an object that has not changed in openness from the walkthrough phase, the action has no affect and the metrics (e.g. Fixed Strict, Success, etc.) do not account for the attempted change in state.
2. Sometimes the openness of an object (particularly drawers) between the walkthrough and unshuffle phase is not noticeably changed, but the object shows up as changed when accessing the object's meta data (openness level between the phases & start energy). I attached an image of a drawer in the walkthrough and unshuffle phases that was tagged as changed by the start energy.
3. When executing pickup {object}, sometimes the wrong object instance is picked up if two objects of the same category are within view. For example, in the image below, the target is the toilet paper roll on the rack, but the roll on the sink is picked up instead. However, this occurs under rare circumstances.

Thank you for all the great developing tools!

[Lucaweihs]

Thanks for the bug report @Gabesarch! We've fixed issues 1&2 for the 2023 challenge, see PR #41 (just merged). For issue 3, we left this as is for now as it's rare and there are generally ways that the agent can restrict its view to only observe the goal object.