New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent merge of same questions on different documents during evaluation #1119
Conversation
…ystack into duplicate_question
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good. Adjusted one doc string slightly and left one comment in the tutorial. Ready to merge once that last comment is resolved
…ystack into duplicate_question
thank you so much @brandenchan for implementing this! |
The issue is raised here (#933). This PR solves this by using a tuple of id and question when aggregating labels. This is done in a few different functions where labels are loaded.
This is a recreation of #1030 and #1100 in order to get a cleaner diff (I rebased and merged locally but Github does not show a clean post-rebase diff)