-
Notifications
You must be signed in to change notification settings - Fork 352
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add app_reviews templates #148
Add app_reviews templates #148
Conversation
templates/app_reviews/templates.yaml
Outdated
reference: Given the review, generate a star rating. | ||
d34e1413-2699-4701-baa2-05d931d012ba: !Template | ||
id: d34e1413-2699-4701-baa2-05d931d012ba | ||
jinja: "What would be the star rating of this review: {{review}}? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This prompt and the one above are ambiguous - the model won't know whether you mean for it to generate the number of stars, a certain number of emojis, etc. I think you need to give the model more instructions as to which you want.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've updated the rating information for both cases. I've added emojis for the star-case, hope that is fine?
…ource into add_app_reviews_templates
templates/app_reviews/templates.yaml
Outdated
f923aeeb-1150-486d-99cd-c4ab93ed5130: !Template | ||
id: f923aeeb-1150-486d-99cd-c4ab93ed5130 | ||
jinja: | ||
"Would you recommend an app to a friend which has a {{star}}-star rating on a scale of 1 to 5 (1 being lowest and 5 being highest)? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This prompt will only result in 5 unique examples - I'm not sure that's enough, I think the model would just memorize the mapping right away.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. I will remove this template then.
templates/app_reviews/templates.yaml
Outdated
9746ce4b-ac58-4dfb-9783-d77c95cb62cf: !Template | ||
id: 9746ce4b-ac58-4dfb-9783-d77c95cb62cf | ||
jinja: | ||
"What would be the \u2B50-rating of this review (\u2B50 being lowest and \u2B50\u2B50\u2B50\u2B50\u2B50 being highest) : {{review}}?\n|||{% if star==5 %}\n\u2B50\ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking in to it, T5's vocab can't tokenize the star emoji:
https://colab.research.google.com/drive/1uXD0f2QdRdk9hMTIW9wF2S_R5tvDperz?usp=sharing
Why not just use * instead?
Thanks! |
Notes:
convert_to_star_rating
. Should a model be able to generate emojis? 🤔Edit
This is from
bert-base-uncased
: