Single Class Span Classification Support

[Steven-Yiran]

Summary

This pull request aims to serve as the starting point to support span classification datasets proposed in #385. Currently, only single-class span classification is supported. One example of a single-class span classification task is Extractive Question Answering.

# Example code snippet
import numpy as np
from cleanlab.experimental.span_classification import find_label_issues

labels = [[0, 0, 1, 1], [1, 1, 0]]
pred_probs = [
    np.array([0.9, 0.9, 0.9, 0.1]),
    np.array([0.1, 0.1, 0.9]),
]

issues = find_label_issues(labels, pred_probs)

Example Usage

An example of using the span_classification module to find label issues in the extractive question-answering dataset SQuAD is provided in a separate repository here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (077f5a9) 96.48% compared to head (7f4f6fd) 96.22%.
Report is 21 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #982      +/-   ##
==========================================
- Coverage   96.48%   96.22%   -0.26%     
==========================================
  Files          73       74       +1     
  Lines        5714     5803      +89     
  Branches     1020     1037      +17     
==========================================
+ Hits         5513     5584      +71     
- Misses        108      130      +22     
+ Partials       93       89       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[aditya1503]

Thanks a lot for your contributions @Steven-Yiran

[Steven-Yiran]

Thanks @aditya1503 @jwmueller for all the helpful guidance and modifications!