No module named 'raft'

[alexfredo]

Hi I have a error when I try to run the script :
C:\AUTOMATIC1111>py script.py
Traceback (most recent call last):
File "C:\AUTOMATIC1111\script.py", line 15, in
from raft import RAFT
ModuleNotFoundError: No module named 'raft'

Apparently I need to install raft but how I add raft to automatic1111 ? thanks

[volotat]

No. You don't need to install raft in automatic1111. It should hook up the 'RAFT' repository when you clone 'SD-CN-Animation'. If it didn't happen for some reason, go to the 'SD-CN-Animation' folder, open CLI and do 'git clone https://github.com/princeton-vl/RAFT.git' command. It should create 'RAFT' folder with all the necessary scripts and it should work fine afterwards.

[alexfredo]

Thanks for raft it's works, but I have another problem when I run the script it's open a window that quickly close and I have just this message in the command : 0it [00:00, ?it/s]

[alexfredo]

in the script I need to write the format like that ? :
INPUT_VIDEO = "C:\AUTOMATIC1111\INPUTVIDEO\1.mp4"
OUTPUT_VIDEO = "C:\AUTOMATIC1111\OUTPUTVIDEO\1.mp4"

[volotat]

Yeah. The format seems fine to me. I suspect It cannot find video file for some reason. Try to move it in 'SD-CN-Animation' folder and set INPUT_VIDEO = "1.mp4". Also make sure that the video is working in any videoplayer.

[alexfredo]

Ok it launched everything and after generate the first picture I have a error :
OpenCV: FFMPEG: tag 0x5634504d/'MP4V' is not supported with codec id 12 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
1%|▋ | 1/117 [00:08<16:53, 8.74s/it]
Traceback (most recent call last):
File "C:\AUTOMATIC1111\SD-CN-Animation\script.py", line 234, in
_, alpha_mask, warped_styled = RAFT_estimate_flow_diff(prev_frame, frame, prev_frame_styled)
File "C:\AUTOMATIC1111\SD-CN-Animation\script.py", line 100, in RAFT_estimate_flow_diff
RAFT_model.load_state_dict(torch.load(args.model))
File "C:\Users\Alex\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 699, in load
with _open_file_like(f, 'rb') as opened_file:
File "C:\Users\Alex\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "C:\Users\Alex\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 211, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'RAFT/models/raft-things.pth'

It's the video format not supported ?

[volotat]

Nope, It cannot find the 'raft-things.pth' model. I forget to mention it in the instruction, you have to set up the RAFT repository as it described here: https://github.com/princeton-vl/RAFT
Basically it just comes down to running "./download_models.sh" to download the models. I would suggest not to creating a special conda environment, but choose what is more convenient for you.

[alexfredo]

it's works ! but the picture become blurry and dark after few minutes here the first pictures :

and the last :

[alexfredo]

Also it's generate the video but I have a error message display in the cmd of automatic1111 : Error running process: C:\AUTOMATIC1111\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
File "C:\AUTOMATIC1111\modules\scripts.py", line 417, in process
script.process(p, *script_args)
File "C:\AUTOMATIC1111\extensions\sd-webui-controlnet\scripts\controlnet.py", line 628, in process
unit = self.parse_remote_call(p, unit, idx)

[alexfredo]

Do you know why it's become more and more darker after each render here another example :

[alexfredo]

Im really impress by the result your script is clearly awesome I can't wait to have fixed the darker images bug

[volotat]

Cannot say anything regarded the web-ui error, but it might be related to the darkening as I never experienced it myself while working on the script.

[alexfredo]

every render it's write :
Error running process: C:\AUTOMATIC1111\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
File "C:\AUTOMATIC1111\modules\scripts.py", line 417, in process
script.process(p, *script_args)
File "C:\AUTOMATIC1111\extensions\sd-webui-controlnet\scripts\controlnet.py", line 628, in process
unit = self.parse_remote_call(p, unit, idx)
File "C:\AUTOMATIC1111\extensions\sd-webui-controlnet\scripts\controlnet.py", line 540, in parse_remote_call
unit.enabled = selector(p, "control_net_enabled", unit.enabled, idx, strict=True)
AttributeError: 'str' object has no attribute 'enabled'

I found someone else have the same error message : AUTOMATIC1111/stable-diffusion-webui#9107

[volotat]

Did you made ControlNet accessible through API in the settings?

[alexfredo]

my controlnet settings :

[volotat]

Ok. It seems like a bug in web-ui. You can do the following: go to modules/api/api.py in web-ui and replace this string (it should be at 275 line)
script_args[alwayson_script.args_from:alwayson_script.args_to] = request.alwayson_scripts[alwayson_script_name]["args"]
with this:

for idx in range(0, min((alwayson_script.args_to - alwayson_script.args_from), len(request.alwayson_scripts[alwayson_script_name]["args"]))):
                        script_args[alwayson_script.args_from + idx] = request.alwayson_scripts[alwayson_script_name]["args"][idx]

Please tell me if it's helped.

[alexfredo]

Thanks it's fixed the webui error message ! but I have always the picture become more and more darker there's another message in the cmd of the script :

[volotat]

Just a warning, you can ignore it. No idea why it is darkening though.

[alexfredo]

It's very frustrating your script have a awesome temporal consistency I really want to use it

[alexfredo]

Im forced to add "--no-half-vae" for generate the video maybe it's because of that ?

[alexfredo]

it's seems to be a problem with the vae AUTOMATIC1111/stable-diffusion-webui#6858

[alexfredo]

It solved the problem I have downloaded the vae for the checkpoint I use, Thanks you for the helps !

[alexfredo]

There's a little bit of flicker but with the davinci resolve deflicker it's fix the problem :)