Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(#1144) allow multilabel records in weak supervision #1233

Merged
merged 26 commits into from Mar 16, 2022

Conversation

leiyre
Copy link
Member

@leiyre leiyre commented Mar 8, 2022

see #1144

@leiyre leiyre requested a review from frascuchon March 8, 2022 15:20
Copy link
Member

@frascuchon frascuchon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Precision, correct and incorrect metrics are missing.

Screenshot 2022-03-08 at 17 13 46

The same dataset with master branch is still working.

Also, labels values in field are misaligned.

@leiyre leiyre requested a review from frascuchon March 10, 2022 11:53
@frascuchon frascuchon added this to In progress in Release via automation Mar 14, 2022
@frascuchon
Copy link
Member

  • No rules defined but the message "This query with this labels are already saved as rule" is shown

Screenshot 2022-03-14 at 10 33 46

@dvsrepo
Copy link
Member

dvsrepo commented Mar 14, 2022

  • No rules defined but the message "This query with this labels are already saved as rule" is shown

Screenshot 2022-03-14 at 10 33 46

@leiyre also the message should be these labels and not this labels. Thanks!!

@leiyre leiyre requested a review from frascuchon March 14, 2022 12:02
@frascuchon
Copy link
Member

Adding a label to an existing rule don't refresh the labes in rules summary

Screenshot 2022-03-14 at 23 06 58

Screenshot 2022-03-14 at 23 07 50

@codecov
Copy link

codecov bot commented Mar 15, 2022

Codecov Report

Merging #1233 (1a688c4) into master (07b895d) will decrease coverage by 0.40%.
The diff coverage is 50.00%.

❗ Current head 1a688c4 differs from pull request most recent head a64b620. Consider uploading reports for the commit a64b620 to get more accurate results

@@            Coverage Diff             @@
##           master    #1233      +/-   ##
==========================================
- Coverage   94.84%   94.43%   -0.41%     
==========================================
  Files         127      127              
  Lines        5449     5591     +142     
==========================================
+ Hits         5168     5280     +112     
- Misses        281      311      +30     
Flag Coverage Δ
pytest 94.43% <50.00%> (-0.41%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/rubrix/server/commons/es_wrapper.py 88.46% <50.00%> (ø)
src/rubrix/client/models.py 89.60% <0.00%> (-7.30%) ⬇️
src/rubrix/client/datasets.py 95.46% <0.00%> (-2.57%) ⬇️
src/rubrix/server/commons/es_helpers.py 84.47% <0.00%> (-1.96%) ⬇️
src/rubrix/server/tasks/text2text/api/model.py 98.43% <0.00%> (-0.20%) ⬇️
...rix/server/tasks/token_classification/api/model.py 98.18% <0.00%> (-0.10%) ⬇️
...brix/server/tasks/text_classification/api/model.py 99.41% <0.00%> (-0.03%) ⬇️
src/rubrix/monitoring/_transformers.py 94.04% <0.00%> (ø)
src/rubrix/server/tasks/commons/dao/dao.py 97.93% <0.00%> (ø)
...rubrix/labeling/text_classification/weak_labels.py 100.00% <0.00%> (ø)
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6899272...a64b620. Read the comment docs.

@frascuchon
Copy link
Member

  1. select an already created rule from rule summary
  2. The precision and correct/incorrect metrics show the rule metrics
  3. change the rule query -> related metrics are reset.
  4. Precision and correct/incorrect metrics are not computed.

Screenshot 2022-03-15 at 15 00 42

Release automation moved this from In progress to Review OK Mar 16, 2022
@frascuchon frascuchon merged commit 7deaf16 into master Mar 16, 2022
Release automation moved this from Review OK to Ready to DEV QA Mar 16, 2022
@frascuchon frascuchon deleted the feat/weak_supervision_multilabel branch March 16, 2022 21:32
@frascuchon frascuchon moved this from Ready to DEV QA to Ready to Release QA in Release Mar 25, 2022
frascuchon added a commit that referenced this pull request Mar 25, 2022
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)
frascuchon added a commit that referenced this pull request Mar 28, 2022
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)
frascuchon added a commit that referenced this pull request Mar 28, 2022
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)

feat(#1144): allow all labels deselection in weak supervision (#1326)

(cherry picked from commit 6e017d4)
@frascuchon frascuchon moved this from Ready to Release QA to Approved Release QA in Release Mar 29, 2022
frascuchon added a commit that referenced this pull request Mar 30, 2022
(cherry picked from commit 1803531)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

feat(#1144): weak multi labels (#1190)

(cherry picked from commit 1ba663f)

fix: computation of correct/incorrect in weak multi label summary (#1237)

(cherry picked from commit 2167296)

feat(#1144): allow multilabel records in weak supervision (#1233)

(cherry picked from commit 7deaf16)

feat(#1144): allow all labels deselection in weak supervision (#1326)

(cherry picked from commit 6e017d4)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Release
Approved Release QA
Development

Successfully merging this pull request may close these issues.

None yet

3 participants