resolution parameter for incoming observations

[kvas7andy]

Hi, everyone!

I found useful to retrieve necessary information form sensors, which I can get from sensors.
For example, to get rgb image observation I do observation['observation']['sensors']['color']['data']
But resolution of output data (arrays) is (128, 128). After trying to change resolution of observations rendering from simulator in sensors.yml as well as via env_config files, I didn't reach any solution.

Could you please tell more about resolution changing (for observations, not for environment rendering via end.render(mode='rgba_array'))?

[angelxuanchang]

Can you clarify what you mean by resolution changing for the observations vs environment rendering? The two are the same and are controlled by external parameters in the command line arguments (such as --width and --height see https://github.com/minosworld/minos/blob/master/config/sim_args.py#L19)

If you look carefully at sensors.yml, you will notice that there is a resize: true. That allows the resolution of the sensor to be overridden and resized by the external command line parameters.

[kvas7andy]

Thank you for clarification!

I work mainly with gym_minos lib, so I use env.render() and observation, as return from env.step(...) function. The concrete usage of --width and --height wasn't clear for me, as well as meaning of resize: true in sensors.yml. As I understood, --width & --height affect both window size, generated by env.render(), as well as rendered images from sensors if it has resize: true flag. Am I right?

[angelxuanchang]

Yes, you are right. Please let us know if you see that this is not the case. You can test out the --width and --height options with the gym demo:
python3 demo.py --env_config objectgoal_suncg_mf --width 256 --height 128

[kvas7andy]

Thank you for your patience and attention.

It required me some time to check if the problem still exists.
So, for equal width and height rendering works fine as well as observation output
For unequal width and height, rendering works fine, but observation output for sensors is given in reversed order (width replaced with height) and image is distorted.
I think, the problem is similar to previous problem which was solved by @msavva, here #9

So the problem still exists.

width = 256
height = 128

The same problem with depth output.

[kvas7andy]

@angelxuanchang
I couldn't find my own workaround, so I would like to ask your help in this problem, stated above (my previous post)

[msavva]

@kvas7andy I am unable to reproduce the issue you observe. Running python3 demo.py --env_config pointgoal_suncg_mf --width 256 --height 128 in minos/gym produces output as below:

Can you confirm that you have cloned a clean copy of the repository and compiled the minos server from scratch? If you have multiple minos repositories, note that you may need to pip uninstall minos and pip install -e . in the code repository you want to use.

[msavva]

@kvas7andy I believe I understand why you observe the issue you reported. You must be pulling out img = observation['observation']['sensors']['color']['data'] and saving the numpy array as an image directly. In order for this image to look correct you will need to reshape such that the height is the first dimension using img = img.reshape((img.shape[1], img.shape[0], img.shape[2])).

[msavva]

@kvas7andy Are you able to use the sensor data as you would expect? If yes, shall we close this issue? I will summarize some of the details of this discussion in a FAQ document so that others can also refer to it.

[kvas7andy]

@msavva it would be great to have Friday to make tests!

[kvas7andy]

@msavva it seems, everything is alright except a new issue, that generated RGB images from SUNCG have crossboard texture in places, where is no object: sky outdoors and behind windows. There were no such problem while rendering pictures via env.render(), inside an opened GUI window. Everything was almost completely dark.

This could be considered as enhancement, to make totally dark, or gradient-coloured outdoor environment.

[msavva]

@kvas7andy the "crossboard" pattern is usually how transparency is visualized by image viewers (i.e. alpha values <1 in RGBA images). Can you copy an output image you get so we can verify whether this is the expected behavior?

[msavva]

@kvas7andy are there any remaining issues with the returned observation images?

[kvas7andy]

plt.imshow() and scipy.misc.imsave() have "crossboard" outputs just in my own viewer. After uploading images in github comment (drag-n-drop) crossboards vanished. Thank you for clarification.