Dataset normalization issue

[InhwanBae]

Hi @Sigta678!

Thank you for your great work and for releasing the code! While looking at your dataset preprocessing code, I found an issue in trajectory normalization. In your code, both the observation and the future trajectory are used to normalize the input data. However, the future trajectory should not be available in the test. If future trajectories are available, the model can implicitly obtain how far agents should go. It seems to be data snooping.

Social-SSL/utils/dataset_convertor/parse_utils_homo_leave1out_scale_perm.py

Lines 270 to 273 in 96f6e5b

	scaler = MinMaxScaler(feature_range=(0, 1))
	scaler.fit(np.append(obsv[:,:,:2].reshape(-1,), pred[:,:,:2].reshape(-1,)).reshape(-1,2))
	obsv[:,:,:2] = scaler.transform(obsv[:,:,:2].reshape(-1,2)).reshape(-1,8,2)
	pred[:,:,:2] = scaler.transform(pred[:,:,:2].reshape(-1,2)).reshape(-1,12,2)

In fact, these issues are frequent in recent models which use their own dataloader. For example, there was a similar issue in AgentFormer(Khrylx/AgentFormer#5). Please let me know if there is anything wrong with my understanding.

Thank you.

[Sigta678]

Hi @InhwanBae

Thank you for your notice!
As you suggest, I try to normalize with the observation data only, and the result is nearly the same. Indeed, there is some performance degrade for true. A possible reason for this is that these datasets are in a fixed view, and thus the error might come from those agents walking around the edge of the scene. Hope this clarifies your point!

I will update the preprocessed dataset soon. If you have more concerns, please let me know.