Evaluation details / reproducing results

[kaanakan]

Hi,

I realized you have used 2 euler steps for KTH, Human3.6M and BAIR datasets. I want to ask what was the reason behind using 2 euler steps instead of 1?

If you have results for the model trained with 1 euler step and tested with 1 euler step on these datasets, can you share it? (I think you shared frame-wise metrics for KTH dataset in table 8.)

To be specific, it would be great if you could share:

• results (PSNR,SSIM,LPIPS,FVD) for SRVP model trained and tested with 1 euler step on BAIR, Human3.6M datasets
• FVD scores for SRVP model trained and tested with 1 euler step on KTH dataset.

Based on the explanations in Table 1, we assume that ours means that SRVP model trained with dt=1/2 and tested with dt=1 and ours-dt/2 means that SRVP model trained and tested with dt=1/2. Is this true?

If you share the trained models, we can evaluate them using your scripts to save you the trouble :)

Thanks in advance.

[edouardelasalles]

Hi,

1. We used two euler steps instead of one mainly because it yields better results when increasing generation frame rate at test time, as explained in Appendix F.
2. For KTH, we indeed provide results for models trained and tested with Δt=1 in Table 8 for PSNR, SSIM and LPIPS, and in Table 10 (Ours / VGG column) for FVD.
   For the remaining datasets, we do not have trained models with Δt=1. If you want to train them yourself, you can just change the n_euler_steps hyperparameter to 1 in our code and keep the other hyperparameters as is.
3. In Table 1, "Ours" is the model trained and tested with the Δt specified in the sub-section dedicated to each dataset in the experiment section (e.g. 1/2 for KTH). The "Ours - Δt/2" entries refer to results obtained by doubling the number of euler steps used during training (e.g. 1/4 for KTH).