Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Resolving issue 2853: no answer logic in FARMReader #2856

Merged
merged 11 commits into from Aug 11, 2022
Merged

Resolving issue 2853: no answer logic in FARMReader #2856

merged 11 commits into from Aug 11, 2022

Conversation

sjrl
Copy link
Contributor

@sjrl sjrl commented Jul 20, 2022 •
edited

Related Issue(s): Issue #2853

Proposed changes:

  • Updated FARMReader.eval_on_file to behave the same as FARMReader.eval by adding the options use_no_answer_legacy_confidence and calibrate_conf_scores

Pre-flight checklist

  • I have read the contributors guidelines
  • If this is a code change, I added tests or updated existing ones
  • If this is a code change, I updated the docstrings

@sjrl sjrl requested a review from tstadel July 20, 2022 12:03
@sjrl sjrl changed the title Issue farmreader eval Resolving issue 2853: no answer logic in FARMReader Jul 20, 2022
@tstadel tstadel mentioned this pull request Jul 21, 2022
Closed
1 task
@sjrl sjrl marked this pull request as ready for review August 9, 2022 15:11
@sjrl sjrl requested review from a team as code owners August 9, 2022 15:11
@sjrl
Copy link
Contributor Author

sjrl commented Aug 10, 2022

@tstadel This PR is ready for review.

tstadel
tstadel approved these changes Aug 10, 2022
View reviewed changes
Copy link
Member

@tstadel tstadel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I left a minor comment about pointing out that regarding ranking no_answers are affected by use_confidence_scores in particular.

haystack/nodes/reader/farm.py Show resolved Hide resolved
@sjrl sjrl mentioned this pull request Aug 11, 2022
Closed
@sjrl sjrl merged commit 44e2b1b into master Aug 11, 2022
@sjrl sjrl deleted the issue_farmreader_eval branch August 11, 2022 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brandenchan brandenchan brandenchan left review comments

@tstadel tstadel tstadel approved these changes

Assignees
No one assigned
Labels
topic:eval topic:reader type:bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@sjrl @brandenchan @tstadel @ZanSara