-
Notifications
You must be signed in to change notification settings - Fork 19
[GSOC]: Compare TFCO with mindiff on text data. #27
Comments
Where should we submit the result? Thank you! |
You can add your colabs here for review. |
Hello @bhaktipriya i have completed the given starter task, |
This is excellent @YASH-GU24! Very well documented and clean. Results are presented very systematically. Thanks a ton for working on this! As a final step, I'd like you to add a few more analyses/experiments to the colab. You can add new sections to the colab and reuse the set up you already have.
|
Hello @bhaktipriya , |
Thanks, Yash. I meant adding new constraints to the model. A constraint that explicitly says FPR of each slice Muslim, Jewish, Christian, Hindu, Buddhist, Atheist is less than 0.008. Experiment with this number(make it 0.002 etc) if there's no solution at 0.008 Below is the mindiff scores from your colab, where overall is 0.008, and group wise it's either higher than 0.008(jewish, muslim) or lower(for others). Your goal is to train a model that has good overall fpr with the constrain that FPR of all groups is less than atleast 0.009(to beat mindiff score of Jewish slice). Hope this helps and thanks a lot for working on it! |
@YASH-GU24 thanks for all the good work. Please send your proposals to us. Emails are in the contributor document. |
Hello @bhaktipriya, https://colab.research.google.com/drive/1Di4FgK0ox8q0w97EtmK8SxUj9LOmDWO5?usp=sharing |
TFCO is a technique for constrained optimisation that can be used to make models fairer.
TFCO colab with Fairness indicators: https://colab.sandbox.google.com/github/tensorflow/fairness-indicators/blob/master/g3doc/tutorials/Fairness_Indicators_TFCO_CelebA_Case_Study.ipynb#scrollTo=idY3Uuk3yvty for fairness.
Write a colab for TFCO that uses the text toxicity dataset as we did for mindiff here https://colab.sandbox.google.com/github/tensorflow/model-remediation/blob/master/docs/min_diff/tutorials/min_diff_keras.ipynb. You can extend the mindiff colab and add a TFCO section if you like.
Please use the same baseline model as used in the mindiff example.
Please ensure that the train/test split is same for mindiff and for TFCO.
Compare performance of TFCO and mindiff, and understand where mindiff outperforms TFCO and vice versa.
The text was updated successfully, but these errors were encountered: