Weird results when use Segmentation Pass for inference

[luuil]

RobustVideoMatting/inference.py

Line 127 in f8a26e2

fgr, pha, *rec = model(src, *rec, downsample_ratio)

I changed this line to use Segmentation Pass. (use the pretrained weights rvm_mobilenetv3.pth)

pha, *rec = model(src, *rec, segmentation_pass=True)
fgr = src * pha

But I got weird mask results, something like this, why?

[PeterL1n]

Try pha.sigmoid(), aka fgr = src * pha.sigmoid(). Because the segmentation pass outputs logits. You need a sigmoid to convert it to 0-1 range.

[luuil]

Thank you for your prompt reply. You are right, now the result is correct.

[FengMu1995]

is it necessary to change "pha.sigmoid()" for the train module
"""
pred_seg = self.model_ddp(true_img, segmentation_pass=True)[0]
loss = segmentation_loss(pred_seg, true_seg)
"""

[vacing]

is it necessary to change "pha.sigmoid()" for the train module

it seems not necessary

https://zhang-yang.medium.com/how-is-pytorchs-binary-cross-entropy-with-logits-function-related-to-sigmoid-and-d3bd8fb080e7