Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Weak Supervision] Enable weak supervision for multi_label text classification #1144

Closed
3 tasks done
frascuchon opened this issue Feb 14, 2022 · 1 comment · Fixed by #1326
Closed
3 tasks done

[Weak Supervision] Enable weak supervision for multi_label text classification #1144

frascuchon opened this issue Feb 14, 2022 · 1 comment · Fixed by #1326
Assignees
@frascuchon @dcfidalgo @leiyre
Projects
Release

Comments

@frascuchon
Copy link
Member

frascuchon commented Feb 14, 2022 •
edited

  1. Weak supervision with multi_labels rules (CLIENT)
  2. Define rules with multi_label classes (UI + SERVER)

Tasks

  • Enable multi label selection for "Define Rules" (API and UI)
  • Adapt precision and correct/incorrect metrics (UII and API)
  • Adapt WeakLabelMatrix and LabelModel (Client)
@frascuchon frascuchon self-assigned this Feb 14, 2022
@frascuchon frascuchon added this to Backlog in Release via automation Feb 14, 2022
@frascuchon frascuchon moved this from Backlog to Planified in Release Feb 15, 2022
frascuchon added a commit that referenced this issue Feb 16, 2022
@frascuchon
feat(#1144): handle multiple labels for tc labeling rules
b07a287
@frascuchon frascuchon mentioned this issue Feb 16, 2022
Merged
frascuchon added a commit that referenced this issue Feb 20, 2022
@frascuchon
Feat/multi labels labeling rules (#1166)
1803531 
* feat(#1144): handle multiple labels for tc labeling rules

* test: improve labeling rules tests

* fix: labels with dot
@dcfidalgo dcfidalgo mentioned this issue Feb 23, 2022
Merged
dcfidalgo pushed a commit that referenced this issue Feb 28, 2022
feat(#1144): weak multi labels (#1190)
1ba663f 
* feat: adapt rules to hold multi labels

* feat: create WeakMultiLabels

* feat: implement summary for WeakMultiLabels

* chore: compute set of labels

* chore: refactor + fix stuff

* test: adapt test due to refactor

* test: complete tests

* chore: add weak multi label class to init

* docs: small fix to docstrings

* docs: add WeakMultiLabels to reference docs

* docs: small refactor python reference

* fix: get labels from rules and annotations

* chore: remove SingleLabelError

* test: fix test
@leiyre leiyre mentioned this issue Mar 8, 2022
Merged
frascuchon added a commit that referenced this issue Mar 16, 2022
@leiyre @frascuchon
feat(#1144) allow multilabel records in weak supervision (#1233)
7deaf16 
* show button in sidebar

* allow multilabel

* show labels in summary

* fix lint

* remove unused methods

* show labels in summary table

* remove console.log

* use singular or plural for info message

* change info when dataset is multilabel

* fix fetch metrics call

* uri from labels array

* fix metrics URL

* change copy depend to selected labels number

* fix show "already saved rule" when are selected labels

* fix #1236

* prevent undefined rule

* fix "already saved rule" copy

* reset store logic

* fix: return stored rule

* allow multiple selection when the results are empty

* refactor: force document refresh after rule creation

* fix update metrics when rule has default labels

* labels visualization in rules summary

* display label block

* fix: small fix for load_rules in client

Co-authored-by: Francisco Aranda <francisco@recogn.ai>
Co-authored-by: dcfidalgo <david@recogn.ai>
@frascuchon frascuchon moved this from Planified to In progress in Release Mar 16, 2022
@frascuchon frascuchon moved this from In progress to Ready to DEV QA in Release Mar 21, 2022
@frascuchon frascuchon moved this from Ready to DEV QA to Approved Release QA in Release Mar 25, 2022
@frascuchon frascuchon moved this from Approved Release QA to Ready to Release QA in Release Mar 25, 2022
frascuchon added a commit that referenced this issue Mar 25, 2022
@frascuchon
feat(#1144): weak supervision for multilabel datasets (#1166)
6f5745c 
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)
Release automation moved this from Ready to Release QA to Ready to DEV QA Mar 28, 2022
@frascuchon frascuchon moved this from Ready to DEV QA to Ready to Release QA in Release Mar 28, 2022
@Amelie-V Amelie-V reopened this Mar 28, 2022
Release automation moved this from Ready to Release QA to In progress Mar 28, 2022
@Amelie-V
Copy link
Member

Amelie-V commented Mar 28, 2022 •
edited by leiyre

Single/multi labels

  • Allow deselecting all labels while I'm creating or editing a rule (show Save disabeled)
  • Change are to is : "This query with these labels is already saved as rule"

@leiyre leiyre mentioned this issue Mar 28, 2022
Merged
frascuchon added a commit that referenced this issue Mar 28, 2022
@frascuchon
feat(#1144): weak supervision for multilabel datasets (#1166)
5b0fa7c 
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)
Release automation moved this from In progress to Ready to DEV QA Mar 28, 2022
frascuchon pushed a commit that referenced this issue Mar 28, 2022
@leiyre
feat(#1144): allow all labels deselection in weak supervision (#1326)
6e017d4 
closes #1144
frascuchon added a commit that referenced this issue Mar 28, 2022
@frascuchon
feat(#1144): weak supervision for multilabel datasets (#1166)
d832595 
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)

feat(#1144): allow all labels deselection in weak supervision (#1326)

(cherry picked from commit 6e017d4)
@frascuchon frascuchon moved this from Ready to DEV QA to Ready to Release QA in Release Mar 28, 2022
@frascuchon frascuchon moved this from Ready to Release QA to Approved Release QA in Release Mar 30, 2022
frascuchon added a commit that referenced this issue Mar 30, 2022
@frascuchon
feat(#1144): weak supervision for multilabel datasets (#1166)
fd95bae 
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)

feat(#1144): allow all labels deselection in weak supervision (#1326)

(cherry picked from commit 6e017d4)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@frascuchon frascuchon

@dcfidalgo dcfidalgo

@leiyre leiyre

Labels
None yet
Projects
No open projects
Release
Approved Release QA
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(#1144): allow all labels deselection in weak supervision argilla-io/argilla
4 participants
@frascuchon @dcfidalgo @leiyre @Amelie-V