Ordinal Labels

[vikasraykar]

Is it possible to support aggregation of ordinal labels as a part of this toolkit via this reduction algorithm.

• Labels are categorical but have an ordering defined 1 < ... < K.
• The K class ordinal labels are transformed into K−1 binary class label data.
• Each of the binary task is then aggregated via crowdkit to estimate Pr[yi > c] for c = 1,...,K −1.
• The probability of the actual class values can then be obtained as Pr[yi = c] = Pr[yi > c−1 and yi ≤ c] = Pr[yi > c−1]−Pr[yi > c].
• The class with the maximum probability is assigned to the instance

[pilot7747]

Hello @vikasraykar!

Thank you for posting this issue. I think it's possible. Do you have any paper in mind that provides some details about this algorithm? I wonder how to set constraints for the aggregation to output Pr[yi > c1] > Pr[yi > c2] in case when c1 < c2. I'm not sure this constraint will be satisfied if we just run, for instance, separate DS aggregations for each of the binary classification tasks.

[vikasraykar]

A Simple Approach to Ordinal Classification
Eibe Frank and Mark Hall
https://www.cs.waikato.ac.nz/~eibe/pubs/ordinal_tech_report.pdf