Lab2: Part2_Debiasing Test Set Bias

[rvh72]

Hello,
I have a question on test set bias of the standard CNN model. After running cell #13 in Part2_Debiasing.ipynb, I see this histogram:

Am I correct in interpreting that avg test set accuracy is ~65% for Light Female, ~70% for Light Male, ~82% for Dark Female, and ~90% for Dark Male?

If training set has majority of data on light skinned female, I expected test set accuracy to be higher for that category. So, results of above hisogram are counter intuitive.

What am I missing?

--Rahul

[markusjohnsson]

Yeah I'm getting unexpected results too, did you find an explanation @rvh72 ? Interestingly I'm not getting the same distribution as you, but I'm still seeing lowest probability (?) for Light Female.

[rvh72]

No, haven't found any explanation. This sort of a base result does not indicate any need for de-biasing the dataset. What distribution did you get?