Skip to content
Permalink
Browse files

Added interactive converter.

With interactive converter you can change any parameter of any frame and see the result in real time.

Converter: added motion_blur_power param.
Motion blur is applied by precomputed motion vectors.
So the moving face will look more realistic.

RecycleGAN model is removed.

Added experimental AVATAR model. Minimum required VRAM is 6GB (NVIDIA), 12GB (AMD)
Usage:
1) place data_src.mp4 10-20min square resolution video of news reporter sitting at the table with static background,
   other faces should not appear in frames.
2) process "extract images from video data_src.bat" with FULL fps
3) place data_dst.mp4 video of face who will control the src face
4) process "extract images from video data_dst FULL FPS.bat"
5) process "data_src mark faces S3FD best GPU.bat"
6) process "data_dst extract unaligned faces S3FD best GPU.bat"
7) train AVATAR.bat stage 1, tune batch size to maximum for your card (32 for 6GB), train to 50k+ iters.
8) train AVATAR.bat stage 2, tune batch size to maximum for your card (4 for 6GB), train to decent sharpness.
9) convert AVATAR.bat
10) converted to mp4.bat

updated versions of modules
  • Loading branch information...
iperov committed Aug 24, 2019
1 parent 3f0bf2e commit 407ce3b1ca2690e7436d18beae4fe3c5c6aa5757
Showing with 2,393 additions and 1,658 deletions.
  1. +35 −0 converters/ConvertAvatar.py
  2. +392 −0 converters/ConvertMasked.py
  3. +0 −50 converters/Converter.py
  4. +0 −61 converters/ConverterAvatar.py
  5. +317 −0 converters/ConverterConfig.py
  6. +0 −50 converters/ConverterImage.py
  7. +0 −436 converters/ConverterMasked.py
  8. +6 −0 converters/FrameInfo.py
  9. +4 −4 converters/__init__.py
  10. BIN doc/manual_en_google_translated.docx
  11. BIN doc/manual_en_google_translated.pdf
  12. BIN doc/manual_ru.pdf
  13. BIN doc/manual_ru_source.docx
  14. +6 −3 facelib/FaceType.py
  15. +8 −5 facelib/LandmarksProcessor.py
  16. +5 −10 imagelib/__init__.py
  17. +7 −141 imagelib/blur.py
  18. +21 −6 imagelib/common.py
  19. +4 −12 interact/interact.py
  20. +8 −1 joblib/SubprocessFunctionCaller.py
  21. +36 −28 joblib/SubprocessorBase.py
  22. +7 −9 main.py
  23. +447 −209 mainscripts/Converter.py
  24. +138 −0 mainscripts/ConverterScreen/ConverterScreen.py
  25. +1 −0 mainscripts/ConverterScreen/__init__.py
  26. BIN mainscripts/ConverterScreen/gfx/sand_clock_64.png
  27. BIN mainscripts/gfx/help_converter_face_avatar.jpg
  28. BIN mainscripts/gfx/help_converter_face_avatar_source.psd
  29. BIN mainscripts/gfx/help_converter_masked.jpg
  30. BIN mainscripts/gfx/help_converter_masked_source.psd
  31. +42 −37 models/ModelBase.py
  32. +740 −0 models/Model_AVATAR/Model.py
  33. 0 models/{Model_RecycleGAN → Model_AVATAR}/__init__.py
  34. +20 −12 models/Model_DF/Model.py
  35. +19 −11 models/Model_H128/Model.py
  36. +19 −11 models/Model_H64/Model.py
  37. +20 −12 models/Model_LIAEF128/Model.py
  38. +0 −482 models/Model_RecycleGAN/Model.py
  39. +30 −26 models/Model_SAE/Model.py
  40. +26 −22 nnlib/nnlib.py
  41. +2 −2 requirements-colab.txt
  42. +2 −2 requirements-cpu.txt
  43. +2 −2 requirements-cuda.txt
  44. +2 −2 requirements-opencl.txt
  45. +16 −12 samplelib/SampleProcessor.py
  46. +11 −0 utils/os_utils.py
@@ -0,0 +1,35 @@
import cv2
import numpy as np

import imagelib
from facelib import FaceType, LandmarksProcessor
from utils.cv2_utils import *

def process_frame_info(frame_info, inp_sh):
img_uint8 = cv2_imread (frame_info.filename)
img_uint8 = imagelib.normalize_channels (img_uint8, 3)
img = img_uint8.astype(np.float32) / 255.0

img_mat = LandmarksProcessor.get_transform_mat (frame_info.landmarks_list[0], inp_sh[0], face_type=FaceType.FULL_NO_ALIGN)
img = cv2.warpAffine( img, img_mat, inp_sh[0:2], flags=cv2.INTER_CUBIC )
return img

def ConvertFaceAvatar (cfg, prev_temporal_frame_infos, frame_info, next_temporal_frame_infos):
inp_sh = cfg.predictor_input_shape

prev_imgs=[]
next_imgs=[]
for i in range(cfg.temporal_face_count):
prev_imgs.append( process_frame_info(prev_temporal_frame_infos[i], inp_sh) )
next_imgs.append( process_frame_info(next_temporal_frame_infos[i], inp_sh) )
img = process_frame_info(frame_info, inp_sh)

prd_f = cfg.predictor_func ( prev_imgs, img, next_imgs )

out_img = np.clip(prd_f, 0.0, 1.0)

if cfg.add_source_image:
out_img = np.concatenate ( [cv2.resize ( img, (prd_f.shape[1], prd_f.shape[0]) ),
out_img], axis=1 )

return (out_img*255).astype(np.uint8)

Large diffs are not rendered by default.

This file was deleted.

This file was deleted.

0 comments on commit 407ce3b

Please sign in to comment.
You can’t perform that action at this time.