Next (#318)

* renaming and restructuring (#282)

* Renaming and restructuring

* Renaming and restructuring

* Renaming and restructuring

* Fix gender detection

* Implement distance to face debugger

* Implement distance to face debugger part2

* Implement distance to face debugger part3

* Mark as next

* Fix reference when face_debugger comes first

* Use official onnxruntime nightly

* CUDA on steroids

* CUDA on steroids

* Add some testing

* Set inswapper_128_fp16 as default

* Feat/block until post check (#292)

* Block until download is done

* Introduce post_check()

* Fix webcam

* Update dependencies

* Add --force-reinstall to installer

* Introduce config ini (#298)

* Introduce config ini

* Fix output video encoder

* Revert help listings back to commas, Move SSL hack to download.py

* Introduce output-video-preset which defaults to veryfast

* Mapping for nvenc encoders

* Rework on events and non-blocking UI

* Add fast bmp to temp_frame_formats

* Add fast bmp to temp_frame_formats

* Show total processing time on success

* Show total processing time on success

* Show total processing time on success

* Move are_images, is_image and is_video back to filesystem

* Fix some spacings

* Pissing everyone of by renaming stuff

* Fix seconds output

* feat/video output fps (#312)

* added output fps slider, removed 'keep fps' option (#311)

* added output fps slider, removed 'keep fps' option

* now uses passed fps instead of global fps for ffmpeg

* fps values are now floats instead of ints

* fix previous commit

* removed default value from fps slider

this is so we can implement a dynamic default value later

* Fix seconds output

* Some cleanup

---------

Co-authored-by: Ran Shaashua <47498956+ranshaa05@users.noreply.github.com>

* Allow 0.01 steps for fps

* Make fps unregulated

* Make fps unregulated

* Remove distance from face debugger again (does not work)

* Fix gender age

* Fix gender age

* Hotfix benchmark suite

* Warp face normalize (#313)

* use normalized kp templates

* Update face_helper.py

* My 50 cents to warp_face()

---------

Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

* face-swapper-weight (#315)

* Move prepare_crop_frame and normalize_crop_frame out of apply_swap

* Fix UI bug with different range

* feat/output video resolution (#316)

* Introduce detect_video_resolution, Rename detect_fps to detect_video_fps

* Add calc_video_resolution_range

* Make output resolution work, does not auto-select yet

* Make output resolution work, does not auto-select yet

* Try to keep the origin resolution

* Split code into more fragments

* Add pack/unpack resolution

* Move video_template_sizes to choices

* Improve create_video_resolutions

* Reword benchmark suite

* Optimal speed for benchmark

* Introduce different video memory strategies, rename max_memory to max… (#317)

* Introduce different video memory strategies, rename max_memory to max_system_memory

* Update readme

* Fix limit_system_memory call

* Apply video_memory_strategy to face debugger

* Limit face swapper weight to 3.0

* Remove face swapper weight due bad render outputs

* Show/dide logic for output video preset

* fix uint8 conversion

* Fix whitespace

* Finalize layout and update preview

* Fix multi renders on face debugger

* Restore less restrictive rendering of preview and stream

* Fix block mode for model downloads

* Add testing

* Cosmetic changes

* Enforce valid fps and resolution via CLI

* Empty config

* Cosmetics on args processing

* Memory workover (#319)

* Cosmetics on args processing

* Fix for MacOS

* Rename all max_ to _limit

* More fixes

* Update preview

* Fix whitespace

---------

Co-authored-by: Ran Shaashua <47498956+ranshaa05@users.noreply.github.com>
Co-authored-by: Harisreedhar <46858047+harisreedhar@users.noreply.github.com>

README.md

@@ -45,7 +45,10 @@ execution:
   --execution-providers EXECUTION_PROVIDERS [EXECUTION_PROVIDERS ...]                                                choose from the available execution providers (choices: cpu, ...)
   --execution-thread-count [1-128]                                                                                   specify the number of execution threads
   --execution-queue-count [1-32]                                                                                     specify the number of execution queries
-  --max-memory [0-128]                                                                                               specify the maximum amount of ram to be used (in gb)
+
+memory:
+  --video-memory-strategy {strict,moderate,tolerant}                                                                 specify strategy to handle the video memory
+  --system-memory-limit [0-128]                                                                                      specify the amount (gb) of system memory to be used
 
 face analyser:
   --face-analyser-order {left-right,right-left,top-bottom,bottom-top,small-large,large-small,best-worst,worst-best}  specify the order used for the face analyser
@@ -70,15 +73,17 @@ face mask:
 frame extraction:
   --trim-frame-start TRIM_FRAME_START                                                                                specify the start frame for extraction
   --trim-frame-end TRIM_FRAME_END                                                                                    specify the end frame for extraction
-  --temp-frame-format {jpg,png}                                                                                      specify the image format used for frame extraction
+  --temp-frame-format {jpg,png,bmp}                                                                                  specify the image format used for frame extraction
   --temp-frame-quality [0-100]                                                                                       specify the image quality used for frame extraction
   --keep-temp                                                                                                        retain temporary frames after processing
 
 output creation:
   --output-image-quality [0-100]                                                                                     specify the quality used for the output image
   --output-video-encoder {libx264,libx265,libvpx-vp9,h264_nvenc,hevc_nvenc}                                          specify the encoder used for the output video
+  --output-video-preset {ultrafast,superfast,veryfast,faster,fast,medium,slow,slower,veryslow}                       specify the preset used for the output video
   --output-video-quality [0-100]                                                                                     specify the quality used for the output video
-  --keep-fps                                                                                                         preserve the frames per second (fps) of the target
+  --output-video-resolution OUTPUT_VIDEO_RESOLUTION                                                                  specify the resolution used for the output video
+  --output-video-fps OUTPUT_VIDEO_FPS                                                                                specify the frames per second (fps) used for the output video
   --skip-audio                                                                                                       omit audio from the target
 
 frame processors:

facefusion.ini

@@ -0,0 +1,66 @@
+[general]
+source_paths =
+target_path =
+output_path =
+
+[misc]
+skip_download =
+headless =
+log_level =
+
+[execution]
+execution_providers =
+execution_thread_count =
+execution_queue_count =
+
+[memory]
+video_memory_strategy =
+system_memory_limit =
+
+[face_analyser]
+face_analyser_order =
+face_analyser_age =
+face_analyser_gender =
+face_detector_model =
+face_detector_size =
+face_detector_score =
+
+[face_selector]
+face_selector_mode =
+reference_face_position =
+reference_face_distance =
+reference_frame_number =
+
+[face_mask]
+face_mask_types =
+face_mask_blur =
+face_mask_padding =
+face_mask_regions =
+
+[frame_extraction]
+trim_frame_start =
+trim_frame_end =
+temp_frame_format =
+temp_frame_quality =
+keep_temp =
+
+[output_creation]
+output_image_quality =
+output_video_encoder =
+output_video_preset =
+output_video_quality =
+output_video_resolution =
+output_video_fps =
+skip_audio =
+
+[frame_processors]
+frame_processors =
+face_debugger_items =
+face_enhancer_model =
+face_enhancer_blend =
+face_swapper_model =
+frame_enhancer_model =
+frame_enhancer_blend =
+
+[uis]
+ui_layouts =

facefusion/choices.py

@@ -1,8 +1,9 @@
 from typing import List
 
-from facefusion.typing import FaceSelectorMode, FaceAnalyserOrder, FaceAnalyserAge, FaceAnalyserGender, FaceMaskType, FaceMaskRegion, TempFrameFormat, OutputVideoEncoder
-from facefusion.common_helper import create_range
+from facefusion.typing import VideoMemoryStrategy, FaceSelectorMode, FaceAnalyserOrder, FaceAnalyserAge, FaceAnalyserGender, FaceMaskType, FaceMaskRegion, TempFrameFormat, OutputVideoEncoder, OutputVideoPreset
+from facefusion.common_helper import create_int_range, create_float_range
 
+video_memory_strategies : List[VideoMemoryStrategy] = [ 'strict', 'moderate', 'tolerant' ]
 face_analyser_orders : List[FaceAnalyserOrder] = [ 'left-right', 'right-left', 'top-bottom', 'bottom-top', 'small-large', 'large-small', 'best-worst', 'worst-best' ]
 face_analyser_ages : List[FaceAnalyserAge] = [ 'child', 'teen', 'adult', 'senior' ]
 face_analyser_genders : List[FaceAnalyserGender] = [ 'male', 'female' ]
@@ -11,16 +12,19 @@
 face_selector_modes : List[FaceSelectorMode] = [ 'reference', 'one', 'many' ]
 face_mask_types : List[FaceMaskType] = [ 'box', 'occlusion', 'region' ]
 face_mask_regions : List[FaceMaskRegion] = [ 'skin', 'left-eyebrow', 'right-eyebrow', 'left-eye', 'right-eye', 'eye-glasses', 'nose', 'mouth', 'upper-lip', 'lower-lip' ]
-temp_frame_formats : List[TempFrameFormat] = [ 'jpg', 'png' ]
+temp_frame_formats : List[TempFrameFormat] = [ 'jpg', 'png', 'bmp' ]
 output_video_encoders : List[OutputVideoEncoder] = [ 'libx264', 'libx265', 'libvpx-vp9', 'h264_nvenc', 'hevc_nvenc' ]
+output_video_presets : List[OutputVideoPreset] = [ 'ultrafast', 'superfast', 'veryfast', 'faster', 'fast', 'medium', 'slow', 'slower', 'veryslow' ]
 
-execution_thread_count_range : List[float] = create_range(1, 128, 1)
-execution_queue_count_range : List[float] = create_range(1, 32, 1)
-max_memory_range : List[float] = create_range(0, 128, 1)
-face_detector_score_range : List[float] = create_range(0.0, 1.0, 0.05)
-face_mask_blur_range : List[float] = create_range(0.0, 1.0, 0.05)
-face_mask_padding_range : List[float] = create_range(0, 100, 1)
-reference_face_distance_range : List[float] = create_range(0.0, 1.5, 0.05)
-temp_frame_quality_range : List[float] = create_range(0, 100, 1)
-output_image_quality_range : List[float] = create_range(0, 100, 1)
-output_video_quality_range : List[float] = create_range(0, 100, 1)
+video_template_sizes : List[int] = [ 240, 360, 480, 540, 720, 1080, 1440, 2160 ]
+
+execution_thread_count_range : List[int] = create_int_range(1, 128, 1)
+execution_queue_count_range : List[int] = create_int_range(1, 32, 1)
+system_memory_limit_range : List[int] = create_int_range(0, 128, 1)
+face_detector_score_range : List[float] = create_float_range(0.0, 1.0, 0.05)
+face_mask_blur_range : List[float] = create_float_range(0.0, 1.0, 0.05)
+face_mask_padding_range : List[int] = create_int_range(0, 100, 1)
+reference_face_distance_range : List[float] = create_float_range(0.0, 1.5, 0.05)
+temp_frame_quality_range : List[int] = create_int_range(0, 100, 1)
+output_image_quality_range : List[int] = create_int_range(0, 100, 1)
+output_video_quality_range : List[int] = create_int_range(0, 100, 1)

facefusion/common_helper.py

@@ -6,5 +6,9 @@ def create_metavar(ranges : List[Any]) -> str:
 	return '[' + str(ranges[0]) + '-' + str(ranges[-1]) + ']'
 
 
-def create_range(start : float, stop : float, step : float) -> List[float]:
+def create_int_range(start : int, stop : int, step : int) -> List[int]:
+	return (numpy.arange(start, stop + step, step)).tolist()
+
+
+def create_float_range(start : float, stop : float, step : float) -> List[float]:
 	return (numpy.around(numpy.arange(start, stop + step, step), decimals = 2)).tolist()

facefusion/config.py

@@ -0,0 +1,80 @@
+from configparser import ConfigParser
+from typing import Optional, List
+
+from facefusion.filesystem import resolve_relative_path
+
+CONFIG = None
+
+
+def get_config() -> ConfigParser:
+	global CONFIG
+
+	if CONFIG is None:
+		config_path = resolve_relative_path('../facefusion.ini')
+		CONFIG = ConfigParser()
+		CONFIG.read(config_path)
+	return CONFIG
+
+
+def clear_config() -> None:
+	global CONFIG
+
+	CONFIG = None
+
+
+def get_str_value(key : str, fallback : Optional[str] = None) -> Optional[str]:
+	section, option = key.split('.')
+	value = get_config()[section].get(option)
+	if value or fallback:
+		return str(value or fallback)
+	return None
+
+
+def get_int_value(key : str, fallback : Optional[str] = None) -> Optional[int]:
+	section, option = key.split('.')
+	value = get_config()[section].get(option)
+	if value or fallback:
+		return int(value or fallback)
+	return None
+
+
+def get_float_value(key : str, fallback : Optional[str] = None) -> Optional[float]:
+	section, option = key.split('.')
+	value = get_config()[section].get(option)
+	if value or fallback:
+		return float(value or fallback)
+	return None
+
+
+def get_bool_value(key : str, fallback : Optional[str] = None) -> Optional[bool]:
+	section, option = key.split('.')
+	value = get_config()[section].get(option, fallback)
+	if value == 'True' or fallback == 'True':
+		return True
+	if value == 'False' or fallback == 'False':
+		return False
+	return None
+
+
+def get_str_list(key : str, fallback : Optional[str] = None) -> Optional[List[str]]:
+	section, option = key.split('.')
+	value = get_config()[section].get(option)
+	if value or fallback:
+		return [ str(value) for value in (value or fallback).split(' ') ]
+	return None
+
+
+def get_int_list(key : str, fallback : Optional[str] = None) -> Optional[List[int]]:
+	section, option = key.split('.')
+	value = get_config()[section].get(option)
+	if value or fallback:
+		return [ int(value) for value in (value or fallback).split(' ') ]
+	return None
+
+
+def get_float_list(key : str, fallback : Optional[str] = None) -> Optional[List[float]]:
+	section, option = key.split('.')
+	value = get_config()[section].get(option)
+	if value or fallback:
+		return [ float(value) for value in (value or fallback).split(' ') ]
+	return None

facefusion/content_analyser.py

@@ -8,8 +8,9 @@
 
 import facefusion.globals
 from facefusion import wording
-from facefusion.typing import Frame, ModelValue
-from facefusion.vision import get_video_frame, count_video_frame_total, read_image, detect_fps
+from facefusion.typing import Frame, ModelValue, Fps
+from facefusion.execution_helper import apply_execution_provider_options
+from facefusion.vision import get_video_frame, count_video_frame_total, read_image, detect_video_fps
 from facefusion.filesystem import resolve_relative_path
 from facefusion.download import conditional_download
 
@@ -23,8 +24,8 @@
 		'path': resolve_relative_path('../.assets/models/open_nsfw.onnx')
 	}
 }
-MAX_PROBABILITY = 0.80
-MAX_RATE = 5
+PROBABILITY_LIMIT = 0.80
+RATE_LIMIT = 5
 STREAM_COUNTER = 0
 
 
@@ -34,7 +35,7 @@ def get_content_analyser() -> Any:
 	with THREAD_LOCK:
 		if CONTENT_ANALYSER is None:
 			model_path = MODELS.get('open_nsfw').get('path')
-			CONTENT_ANALYSER = onnxruntime.InferenceSession(model_path, providers = facefusion.globals.execution_providers)
+			CONTENT_ANALYSER = onnxruntime.InferenceSession(model_path, providers = apply_execution_provider_options(facefusion.globals.execution_providers))
 	return CONTENT_ANALYSER
 
 
@@ -52,11 +53,11 @@ def pre_check() -> bool:
 	return True
 
 
-def analyse_stream(frame : Frame, fps : float) -> bool:
+def analyse_stream(frame : Frame, video_fps : Fps) -> bool:
 	global STREAM_COUNTER
 
 	STREAM_COUNTER = STREAM_COUNTER + 1
-	if STREAM_COUNTER % int(fps) == 0:
+	if STREAM_COUNTER % int(video_fps) == 0:
 		return analyse_frame(frame)
 	return False
 
@@ -75,7 +76,7 @@ def analyse_frame(frame : Frame) -> bool:
 	{
 		'input:0': frame
 	})[0][0][1]
-	return probability > MAX_PROBABILITY
+	return probability > PROBABILITY_LIMIT
 
 
 @lru_cache(maxsize = None)
@@ -87,17 +88,17 @@ def analyse_image(image_path : str) -> bool:
 @lru_cache(maxsize = None)
 def analyse_video(video_path : str, start_frame : int, end_frame : int) -> bool:
 	video_frame_total = count_video_frame_total(video_path)
-	fps = detect_fps(video_path)
+	video_fps = detect_video_fps(video_path)
 	frame_range = range(start_frame or 0, end_frame or video_frame_total)
 	rate = 0.0
 	counter = 0
 	with tqdm(total = len(frame_range), desc = wording.get('analysing'), unit = 'frame', ascii = ' =', disable = facefusion.globals.log_level in [ 'warn', 'error' ]) as progress:
 		for frame_number in frame_range:
-			if frame_number % int(fps) == 0:
+			if frame_number % int(video_fps) == 0:
 				frame = get_video_frame(video_path, frame_number)
 				if analyse_frame(frame):
 					counter += 1
-			rate = counter * int(fps) / len(frame_range) * 100
+			rate = counter * int(video_fps) / len(frame_range) * 100
 			progress.update()
 			progress.set_postfix(rate = rate)
-	return rate > MAX_RATE
+	return rate > RATE_LIMIT