Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tutorial: Human-in-the-loop weak supervision with skweak #869

Merged
merged 6 commits into from
Jan 10, 2022
Merged

Tutorial: Human-in-the-loop weak supervision with skweak #869

merged 6 commits into from
Jan 10, 2022

Conversation

ruanchaves
Copy link
Contributor

Initial version of a weak supervision tutorial with skweak, modeled after the snorkel tutorial and the skweak documentation.

Let me know if any improvements or standardizations are necessary before accepting this PR.

Following other tutorials, I saved some of the data files inside the rubrix repository.

@codecov
Copy link

codecov bot commented Jan 3, 2022

Codecov Report

Merging #869 (8ebdfc2) into master (bd8aa1c) will not change coverage.
The diff coverage is n/a.

❗ Current head 8ebdfc2 differs from pull request most recent head 577d6e5. Consider uploading reports for the commit 577d6e5 to get more accurate results
Impacted file tree graph

@@           Coverage Diff           @@
##           master     #869   +/-   ##
=======================================
  Coverage   95.50%   95.50%           
=======================================
  Files         109      109           
  Lines        4202     4202           
=======================================
  Hits         4013     4013           
  Misses        189      189           
Flag Coverage Δ
pytest 95.50% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/rubrix/server/datasets/dao.py 97.75% <0.00%> (ø)
src/rubrix/server/commons/es_wrapper.py 94.59% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd8aa1c...577d6e5. Read the comment docs.

Copy link
Contributor

@dcfidalgo dcfidalgo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!! I leave you first comments from a quick read, hopefully later today I will have a more thorough look at it.

docs/tutorials/skweak.ipynb Outdated Show resolved Hide resolved
docs/tutorials/skweak.ipynb Outdated Show resolved Hide resolved
docs/tutorials/skweak.ipynb Show resolved Hide resolved
ruanchaves and others added 6 commits January 10, 2022 21:49
Initial version of a weak supervision tutorial with skweak,
modeled after the snorkel tutorial and the skweak documentation.

Let me know if any improvements and standardizations are necessary
before accepting this PR.

Following other tutorials, I saved some of the data files inside
the rubrix repository.
Adjusts tutorial to the new "Setup" section structure.
Fixes minor grammatical errors in markdown cells.
@dcfidalgo
Copy link
Contributor

Thanks a lot @ruanchaves I pushed some minor changes regarding the title and headings, as well as the image paths.
Merging this now.

@dcfidalgo dcfidalgo merged commit 19a5b72 into argilla-io:master Jan 10, 2022
dvsrepo added a commit that referenced this pull request Jan 11, 2022
* 'master' of https://github.com/recognai/rubrix:
  feat(rules): compute total & ann. coverage before label selection (#916)
  fix(rules): compute annotated coverage when no label properly (#915)
  Tutorial: Human-in-the-loop weak supervision with skweak (#869)
  UI: include affected #records to overall coverage/ann. coverage metrics (#914)
  fix lint build (#913)
  UI: manage precision and rules without annotation coverage (#909)
  fix(#876): process 400 response detail properly (#889)
  feat(rules): allow compute partial query rule metrics (#907)
  fix(security): providing default workspace should pass check (#911)
  UI: reset filters from define rules view (#908)
  UI: Show number of created rules in rules management view (#910)
  UI: drop access to rule name field (#904)
  fix(rules): prevent lost rules with dataset updates (#892)
  fix(datasets): process owner as part of dataset id (#870)
  (UI) Rules summary metrics format (#888)
  UI: Improve code snippet for empty workspace (#886)
  fix(UI): Remove case sensitive when filtering labels (#882)
frascuchon pushed a commit that referenced this pull request Jan 11, 2022
* Tutorial: Human-in-the-loop weak supervision with skweak

Initial version of a weak supervision tutorial with skweak,
modeled after the snorkel tutorial and the skweak documentation.

Let me know if any improvements and standardizations are necessary
before accepting this PR.

Following other tutorials, I saved some of the data files inside
the rubrix repository.

* Response to initial PR comments

Adjusts tutorial to the new "Setup" section structure.
Fixes minor grammatical errors in markdown cells.

* update title + headings

* update img paths

* add tutorial to index

* update heading

Co-authored-by: dcfidalgo <david@recogn.ai>
@ruanchaves ruanchaves deleted the skweak_rubrix_tutorial branch January 18, 2022 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants