Relation Prediction produces error after process 23%, with low rate of label2

[TianzhuQin]

As graph, the Relation Prediction part produces errors after 23%. And the rate of Label2 at the moment is fairly low. On the one hand, I can't see the reason why it's popping out. On the other hand, it is quite strange for the Label2 rate since it is only a prediction. Does this ever happen to you guys?

[larksq]

1. The training might not use the whole dataset after interaction type filtering, hence the early stopping.
2. Low accuracy of label 2, which means no interactions, is typical and shows a cautious relation prediction result for many ambiguous scenarios benefitting the safety of consecutive modules.

[EnnaSachdeva]

1. The training might not use the whole dataset after interaction type filtering, hence the early stopping.

   2. Low accuracy of label 2, which means no interactions, is typical and shows a cautious relation prediction result for many ambiguous scenarios benefitting the safety of consecutive modules.

I'm also getting a similar performance. Early stopping is usually used during the training process, however, here we are using a pre-trained model during relation prediction (after the model has been trained), why is there an early stopping? Can you please elaborate on the 1) point?

[larksq]

We did not implement any early stoppings during training. If you see some scenarios are not being used for training, this is probably because they are filtered by some logic in the data loader. You can search for 'return None' at the function get_instance() in dataset_waymo.py to check each condition.

For example, one of those filters is the agent_type filter. This means if you pass 'vehicle' in the 'agent_type' in the training command, all scenarios that have no vehicles marked to predict will be skipped. And this gets more complicated if you are training for conditional trajectory predictor. Here the loaded relation pickle has only a relation of v2v which requires both two agents to predict to be vehicles. If these conditions are not met, the scenario will be skipped.

[EnnaSachdeva]

Thanks for the clarification.