Multiple Skin Colors

[mynameisfiber]

Right now it seems that the skin color defaults to #C69170 and can only take one value. It may be useful to have multiple default values (since not everyone is white) and check all of them in skinDetect.

[jwagner]

The code certainly does not assume that everyone is white. The RGB values are normalized before looking at their difference:
https://github.com/jwagner/smartcrop.js/blob/master/smartcrop.js#L267

The brightness is then checked separately:
https://github.com/jwagner/smartcrop.js/blob/master/smartcrop.js#L170

In my testing this has worked quite well for various skin colors. Expanding the skin color range further will increase the number of false positives quite a bit so care need to be taken.

Do you have cases where it fails?

[mynameisfiber]

I skip the edge and saturation detection and only do skinDetect to examine it's output over the image. I picked one face as close to #C69170 as I could find and then one that wasn't. The resulting scores looked like:

That being said, with edge and saturation re-enabled, the scoring semes more equal over the faces but I'm assuming that because the skin weight is relatively low:

The test image I used was,

[jwagner]

Thanks for the example.

Red is the skin color detector. So it outputs slightly higher values for the lady on the left than the gentleman on the right side. The reason why it selects the person to the right in the end is because there are more edges on his face (due to the wrinkles from age, the more prominent rim light, his beard, glasses, and possibly less makeup). The in the final scoring the contribution from skin color is skinColor_(detail+skinBias)_importance.

So as far as I'm concerned the algorithm works as intended. The skin color detection if anything prefers the darker skin in this case.

[voronoipotato]

So it prefers old wrinkly people? All I hear is move along smoothskin.

[mynameisfiber]

@jwagner does that explanation still hold even though I disabled edgeDetect and saturationDetect for that first image?

[jwagner]

@mynameisfiber I'm not quite sure what you did there and how you disabled it and arrived at that turquoise debug output. My guess is you just commented the function calls out. The score buffer starts out as a copy of the color buffer so you ended up with the actual rgb values in the 'score buffer', and in this case yes, you have successfully biased the algorithm towards people of a brighter skin complexion. But this is simply because your change did not do what you intended.

If I set the edge and saturation values to zero and set the edge weight to zero (normally the algorithm avoids putting interesting features like skin at the edges, which in case of the woman would be the neck) it crops out the woman.

So yes I think it stills holds.

[mynameisfiber]

Good enough for me! Thanks for being patient... I thought it was worthwhile to investigate this since this project has 4,117 stars and the potential for being biased. Maybe a comment in the code where you select the skin color would be handy to help people select a color and understand where potential biases could?

[jwagner]

I plan to add a FAQ entry or something as this is not the first time the subject has come up that explains how it works, that it tries to be fairly unbiased but also that it's clearly not perfect.