Reproduced results lower than the reported ones

[Ze-Yang]

Hi, I directly ran the released codes without any modification. However, I found that the obtained results are lower than the reported ones by >1 percent point, especially the 10-1 setting with a large gap on the base (0-10) classes.

Relevant log files are provided for your reference. Could you advise the possible reasons that may cause such a problem? Thanks a lot.

	15-5			15-1			10-1
	0-15	16-20	all	0-15	16-20	all	0-10	11-20	all
Reported	78.8	52.0	72.4	70.6	23.7	59.4	55.4	15.1	34.3
Reproduced	76.7	48.4	70.0	69.0	20.5	57.4	38.0	13.4	26.3

[zhangchbin]

Hi, @Ze-Yang ,
I upload the experimental results for each step at: experimental results, and I will update the files of 10-10s soon, when I can access the server.

Here are some results in my git history:

1. 15-5s setting:

2. 15-5 setting:

My environment is requirements.
I re-train the 15-5s overlapped before, and find that the performance of step-0 plays a key role for the final performance.
I will re-train other settings when I can access the server.

[Ze-Yang]

Hi, I am still not able to reproduce especially the 10-1 setting results. What I obtain is far lower than the reported number. Meanwhile, I observe a strange phenomenon that the background performance drops drastically at step 8 and become 0 at step 9. Do you have a similar problem? I attach the results for all steps for your reference. Thanks.

[zhangchbin]

@Ze-Yang
I'm sorry for the late reply. We have fixed bugs in the repository, and you can find the re-implemented results on VOC and ADE20K. If there is any questions, please feel easy to let us know.
Thanks for your help.