Different results between paper and code test

[EMJian]

Hey,
It's a pretty work. I also conduct some comparison experiments in video frame interpolation. But I have several questions in some details.
I've been trying to run this code and to output results in timelens.
However, in terms of PSNR and SSIM, values of metrics from timelens code are different that from your paper 'Time Lens'. I don't know what's wrong.

Some details are provided as follow,
Dataset: HSERGB, BS-ERGB
Test code: uzh-rpg/rpg_timelens
Evaluation code: rpg_event_based_frame_interpolation_evaluation

Like your paper 'Time Lens', I report PSNR and SSIM for all sequences by skipping 1, 5 and 7 frames respectively, and reconstructing the missing frames.

My results (mean+std):

1. skip 5 frames:

PSNR/SSIM	HSERGB(far)	HSERGB(close)
code(timelens)	31.33±2.55/0.883±0.069	31.81±4.20/0.822±0.108
paper(timelens)	33.13±2.10/0.877±0.092	32.19±4.19/0.839±0.090

2. skip 7 frames:

PSNR/SSIM	HSERGB(far)	HSERGB(close)
code(timelens)	30.05±2.24/0.864±0.065	31.54±6.05/0.844±0.120
paper(timelens)	32.31±2.27/0.869±0.110	31.68±4.18/0.835±0.091

3. skip 1 frame:

PSNR/SSIM	BS-ERGB
code(timelens)	24.03±4.30/0.741±0.153
paper(timelens)	28.56/-

In addition, I noticed and solved this issue for HSERGB dataset, as you mentioned before.
"""
In events_aligned each event file with index n contains events between images with index n-1 and n, i.e. event file 000001.npz contains events between images 000000.png and 000001.png.
"""
So, I deleted event file '000000.npz' of each sequence to make sure that all steps are correct for HSERGB dataset. BS-ERGB dataset are still unchanged.
Other additional things are not done. If I have any mistakes please correct me.
What confuses me is that the results obtained using the code are different from the results of the paper.
Looking forward to your reply.

[wefvdge]

Hi!I have some questions about BS-ERGB dataset.I found the x, y in raw event data is much greater than the resolution of RGB image. So do you find out the correspondence of spatial coordinates between event data and image?thanks a lot!