Real world Example Feedbacks

[ynie]

Hey ZhengPeng, I would like to show you some examples where I think BiRefNet can improve on these real world photos. The algorithm currently works great, but I hope these feedback can make it much better. Thank you again for your hard work.

[ynie]

Example:

General Use Heavy

Photoroom iOS

[ynie]

General Use Heavy:

Photoroom on iOS:

[ZhengPeng7]

Wow, thanks. I appreciate it! Improvements need to be made to the contour areas, where the predicted values seem unconfident, neither 0 nor 1. I will look deep into the reason for it. Again, many thanks :)

[ynie]

no problem. I will keep this issue up to date with more issues I find.

[ZhengPeng7]

Sure, I would love to see more typical samples, but I also hope it doesn't cost you too much time.
BTW do you know if there are some open projects for subject extraction in the Photoroom app? I found this ICIP 2021 work from them, of which codes seem incomplete though.

I can do anything except their private datasets to improve the quality of BiRefNet.

[ynie]

Not that I know of. Do you think this is a dataset issue?

[ynie]

Hey @ZhengPeng7 , I'm trying to send out a patch for the comfyUI node, which weight should I use for the examples above?

Or which one is the general use heavy on Fal?

Thanks!

[ZhengPeng7]

The largest version for general use is currently the best one for images in the wild (the first line).

[ynie]

Hello @ZhengPeng7, just checking in to see if what can I help with the basket example above. If you are actively working on it, do you have an ETA? Thank you so much.

[ZhengPeng7]

Hi, ynie, there were some mistakes in the previous training for this kind of task.
Dichotomous image segmentation is different from the image matting task (samples here) -- GT values are in 0 or 1 vs in 0 ~ 1. That's why the segmentation on the hairs of your cute dog is not good enough. I want to use more matting data and increase the weights of L1 loss instead of only BCE+IoU which are both used for DIS task.
But the white boundary of the predicted results is still solved... In the latest version, results are better but that bad phenomenon still exists, I'm still thinking about it. InSPyReNet seems very good on this regions, I'm trying to learn things from it.

[ynie]

Hey Zhengpeng, thank you so much for the information. Will that improve the basket example above?

[ynie]

Here are two more examples:

Example 1:

Original:

Photoroom:

General Use Heavy:

Example 2:

Original:

Photoroom:

General Use Heavy: