Demo collection script

[yuchen93]

Hi, is the demonstration collection script available somewhere? The one in robosuite repo does not seem to output demos with the right format.

Thanks!

[yuchen93]

I thought the conversion script works on the demos collected with robosuite but the file format and action space limits are still off.

[amandlek]

This link here should be what you need. What version of robosuite (and branch) are you using, and what error are you running into?

[yuchen93]

I am on the offline_study branch of robosuite and collected new data in the PickPlaceCan environment.
I processed the dataset with dataset_states_to_obs.py but the resulting observations do not match with those in robomimic dataset. It only has object-state as the observation key for some reason. If I initialize the environment with robosuite it has the correct observations.

Also, running get_data_set_information.py I got:

Traceback (most recent call last):
  File "get_dataset_info.py", line 134, in <module>
    raise Exception("Dataset should have actions in [-1., 1.] but got bounds [{}, {}]".format(action_min, action_max))
Exception: Dataset should have actions in [-1., 1.] but got bounds [-1.0, 1.7948744758575645]

I am not sure why there is a mismatch between environments.

[amandlek]

This is very strange - is there any way we can reproduce the error on our end? Can you clarify exactly what scripts and commands you ran to collect and postprocess data so that we can try the same things on our end?

[yuchen93]

These are the commands I ran:

In robosuite:

python collect_human_demonstrations.py --environment PickPlaceCan

In robomimic:

python conversion/convert_robosuite.py --dataset ~/robosuite/robosuite/models/assets/demonstrations/1657322008_8549528/demo.hdf5
python dataset_states_to_obs.py --done_mode 0 --dataset ~/robosuite/robosuite/models/assets/demonstrations/1657322008_8549528/demo.hdf5 --output_name low_dim.hdf5
python get_dataset_info.py --dataset ~/robosuite/robosuite/models/assets/demonstrations/1657322008_8549528/low_dim.hdf5

Error message this time:

Traceback (most recent call last):
  File "get_dataset_info.py", line 134, in <module>
    raise Exception("Dataset should have actions in [-1., 1.] but got bounds [{}, {}]".format(action_min, action_max))
Exception: Dataset should have actions in [-1., 1.] but got bounds [-3.7500000000000036, 3.7500000000000036]

[amandlek]

I think this is because of an issue in the robosuite collection script - the action isn't being clipped to [-1, 1] before being taken in the environment (and recorded). Can you please add a action = np.clip(action, -1., 1.) line above this line here?

For some context, robomimic expects recorded demonstration actions to be in the range [-1., 1.] so that we can easily scale network outputs into that range (this usually improves learning performance). The robosuite environment also performs action clipping internally, so if you'd like to avoid collecting data again, you could probably just modify your demonstration hdf5 directly (manually clip the actions and update the hdf5) if you'd like to avoid collecting data again.

Hope this helps! Feel free to reopen the issue if errors persist.

[yuchen93]

OK! I'll clip the actions. But that is not the only issue. The observations I got only have 'object-state', which does not match with ones in the robomimic dataset that include 'robot0_' stuff. Do you know what could be the cause of this?

[amandlek]

I just tried the same sequence of commands as you, and I am getting all of the observations. Here is a screenshot below. As a note, the object observations should be "object", not "object-state" - so this appears to be something wrong in your setup. The EnvRobosuite class's observation function should be replacing the raw "object-state" observation from the robosuite environment and renaming it to "object", so maybe you inadvertently changed that file? The relevant file in robomimic is robomimic/envs/env_robosuite.py

[yuchen93]

OK thanks a lot for checking!

[yuchen93]

Did you do any post-processing of the demo data? The ones I collected are all much longer than those in the robomimic dataset like ~700 steps vs <200 steps. Is it just an interface problem?

[amandlek]

Yup I would strongly suspect it's an interface problem. I'm also not sure whether you are postprocessing the demo data to only keep successes (I don't think the robosuite code does that for you) - that might also play a role in the data quality you're seeing.

As for interface considerations, I'd strongly recommend using a SpaceMouse (3D mouse) over the keyboard interface to collect better quality demos. More info here.