Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add, delete and edit rules from Python client #1884

Merged
merged 74 commits into from Nov 23, 2022

Conversation

frascuchon
Copy link
Member

@frascuchon frascuchon commented Nov 11, 2022

Closes #1855

@frascuchon frascuchon linked an issue Nov 11, 2022 that may be closed by this pull request
@frascuchon frascuchon marked this pull request as draft November 11, 2022 11:30
src/argilla/client/api.py Outdated Show resolved Hide resolved
src/argilla/client/sdk/text_classification/api.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
@ufukhurriyetoglu ufukhurriyetoglu self-assigned this Nov 14, 2022
src/argilla/client/sdk/text_classification/api.py Outdated Show resolved Hide resolved
src/argilla/client/sdk/text_classification/api.py Outdated Show resolved Hide resolved
src/argilla/client/sdk/text_classification/api.py Outdated Show resolved Hide resolved
src/argilla/client/sdk/text_classification/api.py Outdated Show resolved Hide resolved
@dvsrepo
Copy link
Member

dvsrepo commented Nov 16, 2022

Hi @ufukhurriyetoglu great work. For updating the docs, I suggest the following:

  1. Update https://docs.argilla.io/en/latest/guides/techniques/weak_supervision.html to reflect that Rules created from python can (and should) be added to the dataset. This will make everything much more comfortable for users. Also show you can update and delete rules.
  2. Update tutorials https://docs.argilla.io/en/latest/tutorials/notebooks/labelling-textclassification-snorkel-weaksupervision.html and https://docs.argilla.io/en/latest/tutorials/notebooks/labelling-textclassification-sklearn-weaksupervision.html . In this case, I would add the rules we use to the dataset. In this way the tutorials become much more reproducible.

In both cases 1 and 2, test that they are working as expected, and take the opportunity to simplify text if you see the opportunity.

@frascuchon
Copy link
Member Author

Not sure if everything is pushed, but some new tests are failing. Could you please take a look https://github.com/argilla-io/argilla/actions/runs/3479400438/jobs/5817898907?

Copy link
Member

@davidberenstein1957 davidberenstein1957 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ufukhurriyetoglu could you also take a look at the docs under the reference/webapp. They currently only contain information about weak labelling for text classification and I feel token classification should be added.

https://docs.argilla.io/en/latest/reference/webapp/features.html#weak-labelling

@ufukhurriyetoglu ufukhurriyetoglu added Labeling type: enhancement Indicates new feature requests labels Nov 21, 2022
@frascuchon frascuchon marked this pull request as ready for review November 21, 2022 16:18
@ufukhurriyetoglu
Copy link
Contributor

@davidberenstein1957 for the webapp/features I checked the post and it seems related to web UI features. This PR is related to python client and UI does not change. As I understand python side is auto generated, we can check if the added modules reflected.

@dvsrepo dvsrepo changed the title feat: Allow create dataset rules from python client feat: Add, delete and edit rules from Python client Nov 22, 2022
Copy link
Member

@dvsrepo dvsrepo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@ufukhurriyetoglu ufukhurriyetoglu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great! thanks for the review and corrections @dvsrepo

docker-compose.yaml Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
docs/_source/tutorials/notebooks/labeling_rules.csv Outdated Show resolved Hide resolved
docs/_source/tutorials/notebooks/labeling_rules.csv Outdated Show resolved Hide resolved
@dvsrepo dvsrepo self-requested a review November 22, 2022 15:15
Copy link
Member

@dvsrepo dvsrepo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ufukhurriyetoglu, typically after changes a new review should be requested. This looks good on my side.

@frascuchon do you need to take another look? Otherwise please merge

Copy link
Member Author

@frascuchon frascuchon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some comments to improve the python reference documentation

See here

src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
src/argilla/labeling/text_classification/rule.py Outdated Show resolved Hide resolved
ufukhurriyetoglu and others added 25 commits November 23, 2022 09:35
Co-authored-by: Francisco Aranda <francis@argilla.io>
Co-authored-by: Francisco Aranda <francis@argilla.io>
Co-authored-by: Francisco Aranda <francis@argilla.io>
@frascuchon frascuchon force-pushed the feat/1855-weak-supervision-rules-python-client branch from 9733ebb to f2ee442 Compare November 23, 2022 08:35
@codecov
Copy link

codecov bot commented Nov 23, 2022

Codecov Report

Base: 94.59% // Head: 94.56% // Decreases project coverage by -0.02% ⚠️

Coverage data is based on head (fe74fe6) compared to base (0ff2de7).
Patch coverage: 93.87% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1884      +/-   ##
===========================================
- Coverage    94.59%   94.56%   -0.03%     
===========================================
  Files          150      150              
  Lines         7044     7090      +46     
===========================================
+ Hits          6663     6705      +42     
- Misses         381      385       +4     
Flag Coverage Δ
pytest 94.56% <93.87%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/argilla/client/sdk/client.py 94.25% <50.00%> (-2.18%) ⬇️
src/argilla/labeling/text_classification/rule.py 97.05% <95.65%> (-2.95%) ⬇️
src/argilla/client/api.py 97.08% <100.00%> (+0.11%) ⬆️
src/argilla/client/sdk/text_classification/api.py 100.00% <100.00%> (ø)
...c/argilla/client/sdk/text_classification/models.py 100.00% <100.00%> (ø)
...c/argilla/labeling/text_classification/__init__.py 100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@frascuchon frascuchon merged commit d534a29 into develop Nov 23, 2022
@frascuchon frascuchon deleted the feat/1855-weak-supervision-rules-python-client branch November 23, 2022 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement Indicates new feature requests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Weak supervision] Add Rules to Web UI via Python
4 participants