Fail to reproduce the reported performance

[staceycy]

Hi,

Thank you very much for your interesting work.

I have run your code with the default parameters, but fail to get the reported results in the paper. Following are the results on validation set:
0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90
My results 0: 26.37 || 98.25 | 56.94, 51.97, 42.12, 33.52, 25.24, 15.52, 7.76, 3.62, 0.66
Reported results 0: 29.99 || 98.39 | 61.17, 56.08, 48.05, 38.98, 30.13, 19.15, 10.55, 4.81, 0.95

Could you please tell me whether I need to modify any other parameters in your codes?

Thank you very much.

Best,
Stacey

[kylemin]

Hi,

Thank you for your interest.
It might be due to different system specifications. Could you let me know the information of your OS, GPU, and PyTorch version?
I do not think that you need to modify any other parameters than the random seed (args.seed in main.py).
I recommend to test with other seed numbers and repeat the training multiple times.

Thank you.

[staceycy]

Hi @kylemin,

Thank you very much for your reply. After changing the Pytorch version, I can achieve similar results as yours. By the way, is it possible for you to share the feature and annotation for the ActivityNet dataset?

Thank you.

Stacey

[kylemin]

Hi again,
I am glad that you could achieve similar results.
Sure, this is the link to the ActivityNet features we used.
ActivityNet-features
I also uploaded the annotation.json file in the google drive.
Thank you.
Kyle

[staceycy]

Hi Kyle,

Many thanks for your prompt reply.
I will try with the ActivityNet features and get back to you when the training is done.

Thank you again for your kind help.

Best,
Stacey