Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

significance of FutureDecoder's sn_out_type variable? #33

Open
PFery4 opened this issue Jul 10, 2023 · 0 comments
Open

significance of FutureDecoder's sn_out_type variable? #33

PFery4 opened this issue Jul 10, 2023 · 0 comments

Comments

@PFery4
Copy link

PFery4 commented Jul 10, 2023

Dear AgentFormer authors,

I would like to get a better understanding of the AgentFormer model. As I review the source code, one of the parameters that has been rather difficult for me to understand is the sn_out_type attribute of the FutureDecoder. So far, here's what I understood about this variable:

  • it is a string, with default value "scene_norm", but it can also take the values "norm" or "vel" if the model is configured as such (by specifying the parameter value in the cfg file)
  • it alters the behaviour of the decode_traj_ar method of the FutureDecoder, by modifying the format of the output seq_out that is being predicted. seq_out is the sequence of positions that is actually predicted by the model.
  • by default, no modification is performed at all.
  • seq_out is translated into dec_motion, which is the actual variable that is compared with the ground truth for loss computation. This is done independently from the alterations performed by sn_out_type (instead, it is done by pred_type, which, quite clearly, defines whether the model is supposed to predict velocities, positions, or positions aligned with the scene origin).
  • sn_out_type does not alter the behaviour of the code in any other place than in FutureDecoder.

From my observations, I am inclined to believe that sn_out_type defines the format of the ground truth. If the ground truth were to be proposed to the model in a different format than the predicted one (eg, if the ground truth trajectories are expressed as velocities while the model predicts scene aligned positions), then sn_out_type is responsible for ensuring that the predicted sequence is first translated into the right format before we can perform comparison between prediction and ground truth for loss calculation. Is this correct? I am uncertain whether that is right, since I've found that sn_out_type does not alter the processing of the ground truth in the data_generator or the preprocessor classes, which I would expect this to be the case, if my guess about this variable acting as a regulator between ground truth and prediction to be right.

If anyone could help me clarify this, I would be very thankful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant