Recreating the performance from the README's gif?

[kiwih]

Hi there, I am trying to recreate the suggestion from the gif in the README. Using the suggested code in the README, I have the following:

from transformers import RobertaTokenizer, T5ForConditionalGeneration
import os

tokenizer = RobertaTokenizer.from_pretrained("Salesforce/codet5-base")
model = T5ForConditionalGeneration.from_pretrained('Salesforce/codet5-base')

text = """
// convert from one currency to another
"""

input_ids = tokenizer(text, return_tensors="pt").input_ids

# simply generate one code span
generated_ids = model.generate(input_ids, max_length=256)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))

However this does not generate the suggested code, it only gets as far as "public static".
What am I doing wrong?

[yuewang-cuhk]

Hi, the model behind the AI coding assistant GIF demo is the CodeT5-base model fine-tuned on Apex code corpurs. Generally if you want to apply CodeT5 for generating code or summaries, you need to fine-tune the model on your specific downstream tasks instead of directly using the pretrained models. For code summarization, we have released Salesforce/codet5-base-multi-sum which you can directly employ for generating summaries for functions in 6 PLs (Ruby/JavaScript/Go/Python/Java/PHP).

[kiwih]

That's disappointing. I would recommend removing the gif or replacing it with one that is more representative of the models actually present in this repository (i.e. have it show the performance of one of the snap shots). Alternatively, can you share the model that is fine-tuned over Apex?