Bias/fairness quantitative measurements

[VictorSanh]

Two goals:

• have quantitative measurements for the paper's Broader impact section
• reflect these in the model cards when we release checkpoints

4 held out evaluation sets identified by the Evaluation WG:

• jigsaw_toxicity_pred
• crows_pairs
• winogender (AXG in SuperGLUE)
• winobias

At the current state, I see
crows_pairs and winobias prompted, jigsaw_toxicity_pred has an opened PR (#451) we need to check, winogender needs to be prompted.

Workflow:

• prompt those that were not prompted yet
• making sure these were actually cached (@VictorSanh can have this caching step done fairly quickly)
• evaluation (normal and score rank evaluation) or maybe they have some special evaluation?
• when we know which checkpoints exactly to eval, get the final numbers to report

[VictorSanh]

@StellaAthena for some reason, i can't assign you to this issue...

[awebson]

Pasted from Slack discussion for reference:

currently in HF datasets https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/data ; what the Eval WG proposed https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification

The former might have been added to our dataset spreadsheet while the latter just got added to HF. Still, I get a download error for both though.

Also, Winogender and Winobias can use the prompts from Winogrande.

[StellaAthena]

@awebson I am able to download both from Kaggle without issue.

Also, Winogender and Winobias can use the prompts from Winogrande.

It looks like crow pairs is the only one that needs prompting then. I can take care of that today.

[StellaAthena]

Actually, it looks like crows-pairs has a template already.

[Iwontbecreative]

Pasted from Slack discussion for reference:

currently in HF datasets https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/data ; what the Eval WG proposed https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification

The former might have been added to our dataset spreadsheet while the latter just got added to HF. Still, I get a download error for both though.

Yes, I added jigsaw_unintended_unintended to datasets since we were going to need it. Is your error related to the manual download required for the dataset? If yes, I think that is expected since the data is obtained through kaggle rather than direct download.

[stephenbach]

Working on a PR for winogender/winobias copying from winogrande.

[VictorSanh]

@Iwontbecreative just reviewed your PR, sorry we only getting back to it now

[awebson]

@stephenbach:

to clarify, we already have some (I think) nice templates for wino_bias
Did you want them to be replaced with the winogrande ones, have winogrande ones added in, or leave alone?
Sorry for the late reply. This is a judgment call. On the one hand, we want to test the biases of the same coreference prompts we use for the non-bias-measuring evals (i.e., SuperGLUE WSC.fixed and Winogrande XL), but I don't want to dismiss what others have already written so far. So best to do a union.

Except I'd reject coreference prompts that:

• Include a hardcoded example in the template itself. That makes it no longer a zero-shot.
• Use jargons like “coreference” or even “referent”. Most non-linguists have no idea what they mean. We prefer non-jargon like “what does x refer to”. See my examples in super_glue wsc.fixed.

@debajyotidatta would you please open a PR as soon as possible with prompts from your working group?

[stephenbach]

Whoops, wsc273 has nothing to do with this issue. Can't seem to remove the mentions above, but disregard please.

[stephenbach]

Current status of 4 datasets:

• jigsaw_toxicity_pred Add 10 orig task templates for Jigsaw Unintended Bias #451 open, changes made, needs re-review
• crows_pairs: prompt crows pairs #442 merged a while ago
• winogender: Update winogender (super_glue/axg) to match fixed RTE. #482 closed, no changes needed
• winobias: Updated wino bias templates. #488 open, needs review