About the number of input channels

[leijue222]

In paper 3.1 :

First, we increase the number of input channels from 3 to 9 to allow for
the extra trimap. We encode the trimap using Gaussian blurs of the definite
foreground and background masks at three different scales (in a similar way
to the method of [19] in interactive segmentation). This encoding differs from
existing approaches in deep image matting, as they usually encode the trimap as
a single channel with value 1 if foreground, 0.5 for unknown and 0 for background.

I know the output channels is 7 (a=1, F=3, B=3)
But why the input channels is 9?
In you code,I saw the input are image and trimap,then input channels will be 4.
So,why the input channels is 9 in paper 3.1?

[xymsh]

I think the input channel is 3 (rgb) + 6 (blurred trimap) = 9.

In code dataloader.py, the author defined a function called read_trimap. This function first reads the single-channel trimap, then transforms the pure foreground and pure background area into one-hot version. At this time, 1 channel trimap -> 2 channel one-hot trimap (indicating fg and bg area).

After that, trimap_transform function in transforms.py is used to blur the fg and bg at 3 scale levels separately. Thus, 2 channel trimap -> 6 channel trimap (3 for fg, and 3 for bg).

In the end, we can get 6 channel trimap + 3 channel RGB.

[leijue222]

I got it.
Thanks!

[kartikwar]

I think the input channel is 3 (rgb) + 6 (blurred trimap) = 9.

In code dataloader.py, the author defined a function called read_trimap. This function first reads the single-channel trimap, then transforms the pure foreground and pure background area into one-hot version. At this time, 1 channel trimap -> 2 channel one-hot trimap (indicating fg and bg area).

After that, trimap_transform function in transforms.py is used to blur the fg and bg at 3 scale levels separately. Thus, 2 channel trimap -> 6 channel trimap (3 for fg, and 3 for bg).

In the end, we can get 6 channel trimap + 3 channel RGB.

how is the one hot encoding done in the code ?

are you referring to these lines ?

# trimap[trimap_im == 1, 1] = 1
# trimap[trimap_im == 0, 0] = 1

[kartikwar]

if yes wouldn't this binarize the trimap? Ideally trimap should be continous b/w 0 and 1