Generate ground truth for relationship inference network

[kevgeo]

Hi,
In the Synthetically Trained Neural Networks for LearningHuman-Readable Plans from Real-World Demonstrations paper, how did you guys exactly generate ground truth for relationship inference?

Does NDDS provide an algorithm where when producing the synthetic image of multiple cubes, it gives ground truth telling which cube is above or left of another cube? Or do we have to write our own algorithm for ground truth?