Skip to content

uclanlp/corefBias

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
css
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Gender Bias in Coreference Resolution:Evaluation and Debiasing Methods

Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang. NAACL-2018 short paper.

Updates:


If you want to try the WinoBias dataset using allennlp, remember to add "pos_tag != '-'" in line 274 here.


We analyze different resolution systems to understand the gender bias issues lying in such systems. Providing the same sentence to the system but only changing the gender of the pronoun in the sentence, the performance of the systems varies. To demonstrate the gender bias issue, we created a WinoBias dataset.

The dataset is generated by the five authors of this paper. We use the professions from the Labor Force Statistics which show gender stereotypes:

Professions and their percentages of women
Male biased Female biased
supervisor 44 cashier 73
janitor 34 teacher 78
cook 38 nurse 90
mover 18 assistant 85
laborer 3.5 secretary 95
constructor 3.5 auditor 61
chief 27 cleaner 89
developer 20 receptionist 90
carpenter 2.1 clerk 72
manager 43 counselors 73
lawyer 35 designer 54
farmer 22 hairdressers 92
driver 6 attendant 76
salesperson 48 writer 63
physician 38 housekeeper 89
guard 22 baker 65
analyst 41 accountant 61
mechanician 4 editor 52
sheriff 14 librarian 84
CEO 39 sewer 80
(Note: to reduce the ambigous of words, we made some modification of these professions: mechanician --> mechanic, sewer --> tailor, constructor --> construction worker, counselors --> counselor, designers --> designer, hairdressers --> hairdresser)

About

To analyze and remove gender bias in coreference resolution systems

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published