README.md

检测框引导的图像编辑（Det-Guided-Inpainting)

Grounded-SAM-Inpainting 示例:

from paddlemix.appflow import Appflow
from ppdiffusers.utils import load_image
import paddle
task = Appflow(app="inpainting",
               models=["GroundingDino/groundingdino-swint-ogc","Sam/SamVitH-1024","stabilityai/stable-diffusion-2-inpainting"]
               )
paddle.seed(1024)
url = "https://bj.bcebos.com/v1/paddlenlp/models/community/GroundingDino/000000004505.jpg"
image_pil =  load_image(url)
result = task(image=image_pil,prompt="bus",inpaint_prompt="a yellow van")

Input Image	Det Prompt	Generated Mask	Inpaint Prompt	Inpaint Image
	bus		a yellow van

文本检测框引导的图像编辑（ChatAndDet-Guided-Inpainting)

Grounded-SAM-chatglm 示例:

import paddle
from paddlemix.appflow import Appflow
from ppdiffusers.utils import load_image
task = Appflow(app="inpainting",
               models=["THUDM/chatglm-6b","GroundingDino/groundingdino-swint-ogc","Sam/SamVitH-1024","stabilityai/stable-diffusion-2-inpainting"]
               )
paddle.seed(1024)
url = "https://bj.bcebos.com/v1/paddlenlp/models/community/GroundingDino/000000004505.jpg"
image_pil =  load_image(url)
inpaint_prompt = "bus is changed to A school bus parked on the roadside"
prompt = "Given caption,extract the main object to be replaced and marked it as 'main_object'," \
         + "Extract the remaining part as 'other prompt', " \
         + "Return main_object, other prompt in English" \
         + "Given caption: {}.".format(inpaint_prompt)
result = task(image=image_pil,prompt=prompt)

一些效果展示

Input Image	Prompt	Generated Mask	Inpaint Prompt
	bus is changed to A school bus parked on the roadside

文本引导的图像编辑（Text-Guided Image Inpainting)

import paddle
from paddlemix.appflow import Appflow
from PIL import Image
from ppdiffusers.utils import load_image
img_url = "https://paddlenlp.bj.bcebos.com/models/community/CompVis/stable-diffusion-v1-4/overture-creations.png"
mask_url = "https://paddlenlp.bj.bcebos.com/models/community/CompVis/stable-diffusion-v1-4/overture-creations-mask.png"

image = load_image(img_url)
mask_image = load_image(mask_url)
paddle.seed(1024)

prompt = "Face of a yellow cat, high resolution, sitting on a park bench"

app = Appflow(app='inpainting',models=['stabilityai/stable-diffusion-2-inpainting'])
image = app(inpaint_prompt=prompt,image=image,seg_masks=mask_image)['result']

image.save("a_yellow_cat.png")

Input Image	Inpaint Prompt	Mask	Inpaint Image
	Face of a yellow cat, high resolution, sitting on a park bench