AnimateDiff: lama cleaner support, option to hide animatediff accordi…

…ons and little fixes

README.md

@@ -1,12 +1,12 @@
 # Replacer
 
+> **Note: AnimateDiff temporary requites this patch, untill it will be merged https://github.com/continue-revolution/sd-webui-animatediff/pull/499**
+
 Replacer is an extension for [AUTOMATIC1111/stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui). The goal of this extension is to automate objects masking by detection prompt, using [sd-webui-segment-anything](https://github.com/continue-revolution/sd-webui-segment-anything), and img2img inpainting in one easy to use tab. It also useful for batch inpaint, and inpaint in video with stable diffusion, animatediff and controlnet
 
 
 ![](/docs/images/main_screenshot.jpg)
 
-> **Note: AnimateDiff temporary requites this patch, untill it will be merged https://github.com/continue-revolution/sd-webui-animatediff/pull/499**
-
 You also can draw your mask instead of or in addition to detection, and take advantage of convenient HiresFix option, and ControlNet inpainting with preserving original image resolution and aspect ratio
 
 > If you find this project useful, please star it on GitHub!

docs/video.md

@@ -9,15 +9,15 @@ You need to provide a path to your video file, or url in `file://` format. On Wi
 ![](/docs/images/replacer_video_animate_diff.jpg)
 
 ### General advice
-Almost all advanced options work here. Inpaint padding doesn't, because it's ControlNet inpainting
+Almost all advanced options work here. Inpaint padding doesn't, because it's ControlNet inpainting. Lama cleaner in masked content enables CN inpaint_only+lama module instead of inpaint_only
 
 Due to high AnimateDiff's consistency in comparison with *"Frame by frame"* mode you can use high `mask blur` and `mask expand`.
 
 Hires fix doesn't work here, and as I think, it basically can't, because it will decrease the consistency. But you can use the "upscaler for img2img" option - these upscalers work and consistent enough.
 
 To increase consistency between fragments, you can use ControlNet, or try to use `Fragment length` = 0 and set up `Context batch size`, `Stride`, `Overlap`
 
-`Latent scale`, `Fragment length` and `Context batch size` are set up for 12GB VRAM with one additional ControlNet units. If you get OutOfMemort error, decrease them
+`Latent scale`, `Fragment length` and `Context batch size` are set up for 12GB VRAM with one additional ControlNet unit. If you get OutOfMemort error, decrease them
 
 If you know any other good advice, please send them into github issues, I can place them here
 

replacer/extensions/controlnet.py

@@ -128,13 +128,15 @@ def enableInpaintModeForCN(gArgs: GenerationArgs, p, previousFrame):
         if gArgs.animatediff_args.needApplyCNForAnimateDiff and i+1 == len(gArgs.cn_args):
             gArgs.cn_args[i].enabled = True
             gArgs.cn_args[i].module = 'inpaint_only'
+            if gArgs.inpainting_fill == 4: # lama cleaner
+                gArgs.cn_args[i].module += "+lama"
             gArgs.cn_args[i].model = gArgs.animatediff_args.cn_inpainting_model
             gArgs.cn_args[i].weight = gArgs.animatediff_args.control_weight
 
         if not gArgs.cn_args[i].enabled:
             continue
 
-        if not IS_SD_WEBUI_FORGE and gArgs.cn_args[i].module == 'inpaint_only':
+        if not IS_SD_WEBUI_FORGE and gArgs.cn_args[i].module.startswith('inpaint_only'):
             if p.image_mask is not None:
                 mask = p.image_mask
                 if p.inpainting_mask_invert:

replacer/options.py

@@ -152,6 +152,10 @@ def needHideSegmentAnythingAccordions():
     res : bool = shared.opts.data.get(EXT_NAME_LOWER + "_hide_segment_anything_accordions", False)
     return res
 
+def needHideAnimateDiffAccordions():
+    res : bool = shared.opts.data.get(EXT_NAME_LOWER + "_hide_animatediff_accordions", False)
+    return res
+
 def needHideReplacerScript():
     res : bool = shared.opts.data.get(EXT_NAME_LOWER + "_hide_replacer_script", False)
     return res
@@ -227,6 +231,16 @@ def on_ui_settings():
         ).needs_reload_ui()
     )
 
+    shared.opts.add_option(
+        EXT_NAME_LOWER + "_hide_animatediff_accordions",
+        shared.OptionInfo(
+            False,
+            f"Hide AnimateDiff accordions in txt2img and img2img tabs. Useful if you installed it only for {EXT_NAME}",
+            gr.Checkbox,
+            section=section,
+        ).needs_reload_ui()
+    )
+
     shared.opts.add_option(
         EXT_NAME_LOWER + "_hide_replacer_script",
         shared.OptionInfo(

replacer/ui/video_ui.py

@@ -38,6 +38,7 @@ def makeVideoUI(comp: AttrDict):
                     )
                     comp.ad_internal_fps = gr.Number(
                         value=0, precision=0, info="Set 0 to use the same with input video",
+                        minimum=0,
                         label="Internal AD FPS", 
                         elem_id="replacer_ad_internal_fps"
                     )
@@ -118,7 +119,8 @@ def makeVideoUI(comp: AttrDict):
 
                 with gr.Accordion("Help", open=False):
                     gr.Markdown(
-                        "Almost all advanced options work here. Inpaint padding doesn't, because it's ControlNet inpainting\n\n"
+                        "Almost all advanced options work here. Inpaint padding doesn't, because it's ControlNet inpainting. "
+                        "Lama cleaner in masked content enables CN inpaint_only+lama module instead of inpaint_only\n\n"
                         \
                         "Due to high AnimateDiff's consistency in comparison with *\"Frame by frame\"* mode "
                         "you can use high `mask blur` and `mask expand`.\n\n"
@@ -131,7 +133,7 @@ def makeVideoUI(comp: AttrDict):
                         "`Fragment length` = 0 and set up `Context batch size`, `Stride`, `Overlap`\n\n"
                         \
                         "`Latent scale`, `Fragment length` and `Context batch size` are set up for 12GB VRAM with one additional "
-                        "ControlNet units. If you get OutOfMemort error, decrease them\n\n"
+                        "ControlNet unit. If you get OutOfMemort error, decrease them\n\n"
                         \
                         "Read [here](https://github.com/light-and-ray/sd-webui-replacer/blob/master/docs/video.md#animatediff-options) "
                         "about AnimateDiff options\n\n"

replacer/video_animatediff.py

@@ -67,19 +67,19 @@ def getFragments(gArgs: GenerationArgs, video_output_dir: str, totalFragments: i
         frame.save(os.path.join(framesDir, f'frame_{frameInFragmentIdx}.png'))
         try:
             mask = createMask(frame, gArgs).mask
+            mask = limitImageByOneDemention(mask, max(gArgs.width, gArgs.height))
+            mask = applyMaskBlur(mask.convert('RGBA'), gArgs.mask_blur)
+            mask = mask.resize(frame.size)
         except Exception as e:
             if type(e) is not NothingDetectedError:
-                errors.report('***', exc_info=True)
+                errors.report(f'{e} ***', exc_info=True)
             else:
                 print(e)
             if mask is None:
                 blackFilling = Image.new('L', frame.size, 0)
                 mask = blackFilling
             else:
                 mask = extraMaskExpand(mask, 50)
-        mask = limitImageByOneDemention(mask, max(gArgs.width, gArgs.height))
-        mask = applyMaskBlur(mask.convert('RGBA'), gArgs.mask_blur)
-        mask = mask.resize(frame.size)
         mask.save(os.path.join(masksDir, f'frame_{frameInFragmentIdx}.png'))
         frameInFragmentIdx += 1
     yield fragmentPath
@@ -96,7 +96,7 @@ def animatediffGenerate(gArgs: GenerationArgs, video_output_dir: str, result_dir
         gArgs.animatediff_args.fragment_length = len(gArgs.images)
     gArgs.animatediff_args.needApplyCNForAnimateDiff = True
 
-    totalFragments = math.ceil((len(gArgs.images) - 1) / gArgs.animatediff_args.fragment_length)
+    totalFragments = math.ceil((len(gArgs.images) - 1) / (gArgs.animatediff_args.fragment_length - 1))
     if gArgs.animatediff_args.generate_only_first_fragment:
         totalFragments = 1
     shared.state.job_count = 1 + totalFragments

scripts/replacer_main_ui.py

@@ -2,7 +2,7 @@
 import gradio as gr
 from modules import script_callbacks, progress, shared, errors
 from replacer.options import (EXT_NAME, EXT_NAME_LOWER, needHideSegmentAnythingAccordions,
-    getDedicatedPagePath, on_ui_settings,
+    getDedicatedPagePath, on_ui_settings, needHideAnimateDiffAccordions, 
 )
 from replacer.ui.tools_ui import IS_WEBUI_1_5
 from replacer.ui import replacer_tab_ui
@@ -58,11 +58,23 @@ def hideSegmentAnythingAccordions(component, **kwargs):
         getattr(component, 'label', "") == "Segment Anything":
 
         component.visible = False
+        print(f"[{EXT_NAME}] Segment Anything accordion has been hidden")
 
 if needHideSegmentAnythingAccordions():
     script_callbacks.on_after_component(hideSegmentAnythingAccordions)
 
 
+def hideAnimateDiffAccordions(component, **kwargs):
+    if type(component) is gr.Accordion and\
+        getattr(component, 'label', "") == "AnimateDiff":
+
+        component.visible = False
+        print(f"[{EXT_NAME}] AnimateDiff accordion has been hidden")
+
+if needHideAnimateDiffAccordions():
+    script_callbacks.on_after_component(hideAnimateDiffAccordions)
+
+
 script_callbacks.on_before_ui(replacer_tab_ui.initMainUI)
 script_callbacks.on_after_component(replacer_extensions.controlnet.watchControlNetUI)
 script_callbacks.on_after_component(replacer_extensions.soft_inpainting.watchSoftInpaintUI)