remove object

[Arcxml]

Hi, thanks for your code. I have a question, if I want to remove an object (for example I want to remove this blueberry), how should I write the prompt? If I wrote "remove the blueberry" it might not work well.

[juxuan27]

Hi, @Arcxml ! Thank you for your focus. The hugging face space uses the segmentation mask checkpoint, thus it will force a boundary in the mask region. I recommend you use a random segmentation mask checkpoint and add blueberry to the negative prompt. Since it is really hard for the diffusion model to understand remove xxx.

[Arcxml]

hello, thank for your reply. Do you think this is the problem from prompt? Or maybe another prompt will work for my task?

…

---- Replied Message ---- | From | Xuan ***@***.***> | | Date | 03/31/2024 13:09 | | To | ***@***.***> | | Cc | ***@***.***>***@***.***> | | Subject | Re: [TencentARC/BrushNet] remove object (Issue #10) | Hi, @Arcxml ! Thank you for your focus. The hugging face space uses the segmentation mask checkpoint, thus it will force a boundary in the mask region. I recommend you use a random segmentation mask checkpoint and add blueberry to the negative prompt. Since it is really hard for the diffusion model to understand remove xxx. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

[juxuan27]

I think the prompt will influence the results. But as I said, I believe the most important reason is the ckpt used in hugging face space. You can try locally changing the ckpt to the random mask one, which can be downloaded from the Google Drive link we provide. I think this may lead to a better result~

Using the segmentation mask ckpt can successfully remove the blueberry by putting blueberry in the negative prompt. However, it would lead to incoherent results due to the force of generating a boundary.

[Arcxml]

If I put the unnecessary objects in the negative prompt, new other objects will be generated. But I don’t want to create any new objects. I want them to complete the background. Do you have any other suggestions?

[juxuan27]

As mentioned in my previous comments, you need to use random mask ckpt instead of segmentation mask ckpt to remove objects.

[Arcxml]

Okay, thanks a lot！

…

---- Replied Message ---- | From | Xuan ***@***.***> | | Date | 04/02/2024 19:03 | | To | ***@***.***> | | Cc | ***@***.***>***@***.***> | | Subject | Re: [TencentARC/BrushNet] remove object (Issue #10) | As mentioned in my previous comments, you need to use random mask ckpt instead of segmentation mask ckpt to remove objects. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>