Bisenet-FP - Improved weights for masker

[torzdf]

The following pull-request allows for the use of custom trained weights for the Bisenet-FP Mask.

The dataset of 40k faces has been built from a wide variety of posed and in the wild images. These have been manually annotated over several months to handle:

• many varieties of real obstructions
• difficult/varied conditions + poses
• glasses (mask out frames, keep lenses)
• dark sunglasses (mask out frames + lenses)
• multiple targets within a single extracted image.

This dataset will continue to be grown over time and will be used to create new and more improved maskers. Whilst this solution is by no means perfect, it is vastly improved over any existing masking solutions.

NB: This Pull Request does not include the actual weights file. This will be going to Patreons first for a timed exclusive, but the weights will eventually be made available for use in the main code.

Glasses + Sunglasses:

Ignoring the non-target face:

Multiple obstruction types:

[arushishot]

How would I apply this? I followed the directions on Patreon, but couldn't find the ".cache" folder after "mask" to extract the zipped file. Would I have to extract in the "pycache" folder?

And is it automatically applied in Faceswap or do I need to uploaded it or select an option for it? TIA!

[torzdf]

How would I apply this? I followed the directions on Patreon, but couldn't find the ".cache" folder after "mask" to extract the zipped file. Would I have to extract in the "pycache" folder?

And is it automatically applied in Faceswap or do I need to uploaded it or select an option for it? TIA!

No need to add anything to the .cache folder. Patreons got this mask well ahead of time as a perk for membership, but it has long been rolled into the main Faceswap branch. Probably best to delete any files you added to .cache, in case I have made any model updates since then (I honestly can't remember).

You can select the mask either in extract or the mask tool (bisenet-fp). You can configure how the mask works (what it masks) in Settings > Extract > Mask > Bisenet-FP.

You can select the bisenet-fp mask for training in Settings > Train > Loss > Mask Type