Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About testing result #25

Closed
wanboyang opened this issue Mar 8, 2022 · 16 comments
Closed

About testing result #25

wanboyang opened this issue Mar 8, 2022 · 16 comments

Comments

@wanboyang
Copy link

wanboyang commented Mar 8, 2022

Thanks for this amazing work.
Could you provide the inference captions of COCO test set by using the method proposed in this paper.
It is hard to download the features used in this paper due to its large size.
Thanks again.

@wanboyang
Copy link
Author

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086.
It seems different from the performance claimed in paper.
Could you provide the inference captions of COCO test set by using the method proposed in this paper.

@hasky123
Copy link

你好,能否提供一份特征?感激不尽!!

@wanboyang
Copy link
Author

wanboyang commented Mar 15, 2022 via email

@zhangxuying1004
Copy link
Owner

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

关于language model,您有没有使用我上传的的模型参数文件?还是您重新训练的?

@wanboyang
Copy link
Author

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

关于language model,您有没有使用我上传的的模型参数文件?还是您重新训练的?

我使用的是上传的的模型参数文件

@wanboyang
Copy link
Author

decoder = TransformerDecoderLayer(len(text_field.vocab), 54, 3, text_field.vocab.stoi['<pad>'], 'Pre-trained_Models/language_context.pth')

@zhangxuying1004
Copy link
Owner

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

关于language model,您有没有使用我上传的的模型参数文件?还是您重新训练的?

我使用的是上传的的模型参数文件

我check一下。

@wanboyang
Copy link
Author

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

关于language model,您有没有使用我上传的的模型参数文件?还是您重新训练的?

我使用的是上传的的模型参数文件

我check一下。

请问有结果了吗

@zhangxuying1004
Copy link
Owner

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

关于language model,您有没有使用我上传的的模型参数文件?还是您重新训练的?

我使用的是上传的的模型参数文件

我check一下。

请问有结果了吗

您好,我前几天也测试了一下,确实是您说的那个结果,应该是这个Project的bug,抱歉。下面是我使用原始Project测试的结果:
结果
这段时间在宿舍隔离,连接不上我们实验室的服务器了,就专心写毕业论文了。隔离结束后再更新这个Project。

@egoistsh
Copy link

egoistsh commented Apr 9, 2022

你好,能更新代码吗,这个精度确实不对。

@gjc0824
Copy link

gjc0824 commented Apr 13, 2022

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

Hello Wan!I also use this open source code to train the RSTNet. However, at the end of cross-entropy training stage, I only get the CIDEr score about 103, is that normal? How much CIDEr score did you get at the end of cross-entropy training stage? How many epochs did you train during RL stage? Did you use other tricks that can tell me?
Thank you very much! I expect your reply.

@wanboyang
Copy link
Author

Hello Wan!I also use this open source code to train the RSTNet. However, at the end of cross-entropy training stage, I only get the CIDEr score about 103, is that normal? How much CIDEr score did you get at the end of cross-entropy training stage? How many epochs did you train during RL stage? Did you use other tricks that can tell me?
Thank you very much! I expect your reply.

I do not train the RSTNet.

@zhuhaifengaaa
Copy link

zhuhaifengaaa commented May 4, 2022

你好,能更新代码吗,这个精度确实不对。
是啊,这个模型跑的结果比不上论文里面的,我不知道哪出了问题,作者能帮解答一下吗?谢谢了

已更新

@zhangxuying1004
Copy link
Owner

代码已更新,下面是测试结果截图。
image

@zhangxuying1004
Copy link
Owner

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

关于language model,您有没有使用我上传的的模型参数文件?还是您重新训练的?

我使用的是上传的的模型参数文件

我check一下。

请问有结果了吗

已更新

@wanboyang
Copy link
Author

I use the pretrained model provide in this project to test COCO test set, and eval performance is 'BLEU': [0.806718016311047, 0.6540980786273354, 0.5092295283787919, 0.3898375465182736], 'METEOR': 0.29228558925065895, 'ROUGE': 0.585628568332276, 'CIDEr': 1.3206302796307086. It seems different from the performance claimed in paper. Could you provide the inference captions of COCO test set by using the method proposed in this paper.

关于language model,您有没有使用我上传的的模型参数文件?还是您重新训练的?

我使用的是上传的的模型参数文件

我check一下。

请问有结果了吗

已更新

谢谢更新

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants