Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimal snips to run CodeT5 on each task #73

Closed
rocco-fortuna opened this issue Nov 17, 2022 · 2 comments
Closed

Minimal snips to run CodeT5 on each task #73

rocco-fortuna opened this issue Nov 17, 2022 · 2 comments

Comments

@rocco-fortuna
Copy link

Hello, I have spent quite some time trying to run the fine-tuned model you kindly provided, on the clone detection task, with no luck.
Could you provide minimal python scripts to load and run the model on each of the downstream tasks, as was done for ntp and summarization on 🤗?

Thanks a lot!

@rocco-fortuna
Copy link
Author

I am happy to contribute by providing such scripts, but need guidance to make your model work. Your finetuned_models_clone_codet5_base.bin model only ever predicts code pairs in the test set are not clones for me, which yields 0 F1 score, not quite the 97% it should reach according to the paper.

@yuewang-cuhk
Copy link
Contributor

Hi, we've shared the instructions to reproduce the results using our released checkpoints here. Please try it or you can rerun the finetuning experiments. If both options still do not work, please share more detailed logs for us to understand the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants