Results without fringe

[danielgatis]

Hi @Nathanua,

I have a library that makes use of your model.

@alfonsmartinez opened an issue about the model result, please, take a look at here:
danielgatis/rembg#14

Can you figure out how I can achieve this result without the black fringe?

thanks.

[kashyappiyush1998]

In my case, this is green edges with green screen

[xuebinqin]

Hi, Daniel, Thanks for your interests. This could be solved by conducting morphological operations (e.g. erosion) on the binarized maps. The size of erosion disk/square can be determined based on your input image size (e.g. input_size/320) or configured experimentally. If you don't care about the time costs, you can also feed the results of U2-Net to CascadedPSP-Net ( https://github.com/hkchengrex/CascadePSP) for getting the original resolution probability maps.

…

On Tue, Sep 29, 2020 at 3:29 PM Daniel Gatis ***@***.***> wrote: Hi @Nathanua <https://github.com/NathanUA>, I have a library that makes use of your model. @alfonsmartinez <https://github.com/alfonsmartinez> opened an issue about the model result, please, take a look at here: danielgatis/rembg#14 <danielgatis/rembg#14> Can you figure out how I can achieve this result without the black fringe? thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#75>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADSGORO5YD66M6ZXI6RVKYDSIJGTXANCNFSM4R6KACZA> .

-- Xuebin Qin PhD Department of Computing Science University of Alberta, Edmonton, AB, Canada Homepage:https://webdocs.cs.ualberta.ca/~xuebin/

[kashyappiyush1998]

But I want to make a general case model which works on both monotonous and complex background, If I dilate on images that already show good results, wouldn't it cut off my borders.

[kashyappiyush1998]

hkchengrex/CascadePSP#25
Results using Cascade PSP are not good either, as well as processing time skyrockets.

[99991]

It might help to create a trimap from the U-2-Net mask by erosion and dilation. The image and trimap can then be used as input for some alpha matting framework.

But even with a hand-drawn trimap, the result will not perfect because there is some blue chromatic aberration on the rice grains (most noticeable on the top right).

[danielgatis]

Hi thomas @99991

It might help to create a trimap from the U-2-Net mask by erosion and dilation.

Sorry for the dumb question but can you provide the code that generates the trimap in this example?

[99991]

@danielgatis That trimap was hand-drawn, but it is not too difficult to generate it automatically from the output of u2net. Simply threshold the prediction to get approximate foreground and background masks, erode the masks to increase the region where alpha matting methods are allowed to calculate the alpha values and then combine the masks to form a trimap.

Here is some code to do this.

I reduced the size of the input image to not run out of memory.

A solution for a less memory hungry method for alpha matting is described in this paper

There is also this paper, where the authors predict an alpha matte directly from the input image without trimap, but the associated repository has no code at the moment and it would probably not be legal to use the network due to the license of the images it was trained on.

[kashyappiyush1998]

Matting Still does not remove the edges,

I have uploaded a small resized image, on original 1920X1080 green edges are clearly visible

[99991]

@kashyappiyush1998 Maybe you are not doing foreground estimation or your alpha matte is too large for some reason. It should also work with green backgrounds.

See for example this image from https://commons.wikimedia.org/wiki/File:Girl_in_front_of_a_green_background.jpg (licensed under Creative Commons Attribution 2.0 Generic license by User "PictureYouth"). This image is somewhat challenging because there is color spill on the shirt due to reflections from the background and yet it works reasonably well even though u2net does not detect the pants correctly.

(Generated with this hacky code, just for testing, not production ready)

[danielgatis]

Hi @kashyappiyush1998,

I have a project that just pipes the u2net output to the pymatting and I got the following results:

curl -s https://upload.wikimedia.org/wikipedia/commons/c/c2/Girl_in_front_of_a_green_background.jpg | rembg -a > a.png

curl -s https://user-images.githubusercontent.com/35548940/95571670-6f866800-0a46-11eb-9a7c-8272e0229ee6.jpg | rembg -a > b.png