Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wrong predicted computation for multi-label datasets #1527

Closed
frascuchon opened this issue May 31, 2022 · 0 comments · Fixed by #1528
Closed

Wrong predicted computation for multi-label datasets #1527

frascuchon opened this issue May 31, 2022 · 0 comments · Fixed by #1528
Assignees
@frascuchon
Labels
area: server Indicates that an issue or pull request is related to the server type: bug Indicates an unexpected problem or unintended behavior
Projects
Release

Comments

@frascuchon
Copy link
Member

Describe the bug
A record annotated/predicted without labels (a special allowed case for the text classification multi-label problem) does not reflect the predicted ok/ko status in the filters.

Also, the f1 metrics are affected.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
No labeled annotated/predicted records should be included as part of predicted ok/ko metrics.

Screenshots
Screenshot 2022-05-31 at 11 49 35

Environment (please complete the following information):

  • OS [e.g. iOS]:
  • Browser [e.g. chrome, safari]:
  • Rubrix Version [e.g. 0.10.0]: 0.14
  • ElasticSearch Version [e.g. 7.10.2]:
  • Docker Image (optional) [e.g. rubrix:v0.10.0]:

Additional context
@frascuchon frascuchon added type: bug Indicates an unexpected problem or unintended behavior area: server Indicates that an issue or pull request is related to the server labels May 31, 2022
@frascuchon frascuchon self-assigned this May 31, 2022
@frascuchon frascuchon added this to Backlog in Release via automation May 31, 2022
@frascuchon frascuchon moved this from Backlog to Planified in Release May 31, 2022
@frascuchon frascuchon mentioned this issue May 31, 2022
Merged
@frascuchon frascuchon moved this from Planified to In progress in Release Jun 2, 2022
@frascuchon frascuchon moved this from In progress to Pending Review in Release Jun 2, 2022
Release automation moved this from Pending Review to Waiting Release Jun 3, 2022
frascuchon added a commit that referenced this issue Jun 3, 2022
@frascuchon
fix(#1527): check agents instead labels for predicted computation (#…
2d0612d 
…1528)
@frascuchon frascuchon moved this from Waiting Release to Ready to Release QA in Release Jun 6, 2022
frascuchon added a commit that referenced this issue Jun 6, 2022
@frascuchon
fix(#1527): check agents instead labels for predicted computation (#…
ddb839c 
…1528)

(cherry picked from commit 2d0612d)
frascuchon added a commit that referenced this issue Jun 7, 2022
@frascuchon
fix(#1527): check agents instead labels for predicted computation (#…
2f2ee2e 
…1528)

(cherry picked from commit 2d0612d)
@frascuchon frascuchon moved this from Ready to Release QA to Approved Release QA in Release Jun 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@frascuchon frascuchon

Labels
area: server Indicates that an issue or pull request is related to the server type: bug Indicates an unexpected problem or unintended behavior
Projects
No open projects
Release
Approved Release QA
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(#1527): check agents instead labels for predicted computation argilla-io/argilla
1 participant
@frascuchon