Skip to content
Permalink
Browse files

added new XSegEditor !

here new whole_face + XSeg workflow:

with XSeg model you can train your own mask segmentator for dst(and/or src) faces
that will be used by the merger for whole_face.

Instead of using a pretrained segmentator model (which does not exist),
you control which part of faces should be masked.

new scripts:
	5.XSeg) data_dst edit masks.bat
	5.XSeg) data_src edit masks.bat
	5.XSeg) train.bat

Usage:
	unpack dst faceset if packed

	run 5.XSeg) data_dst edit masks.bat

	Read tooltips on the buttons (en/ru/zn languages are supported)

	mask the face using include or exclude polygon mode.

	repeat for 50/100 faces,
		!!! you don't need to mask every frame of dst
		only frames where the face is different significantly,
		for example:
			closed eyes
			changed head direction
			changed light
		the more various faces you mask, the more quality you will get

		Start masking from the upper left area and follow the clockwise direction.
		Keep the same logic of masking for all frames, for example:
			the same approximated jaw line of the side faces, where the jaw is not visible
			the same hair line
		Mask the obstructions using exclude polygon mode.

	run XSeg) train.bat
		train the model

		Check the faces of 'XSeg dst faces' preview.

		if some faces have wrong or glitchy mask, then repeat steps:
			run edit
			find these glitchy faces and mask them
			train further or restart training from scratch

Restart training of XSeg model is only possible by deleting all 'model\XSeg_*' files.

If you want to get the mask of the predicted face (XSeg-prd mode) in merger,
you should repeat the same steps for src faceset.

New mask modes available in merger for whole_face:

XSeg-prd	  - XSeg mask of predicted face	-> faces from src faceset should be labeled
XSeg-dst	  - XSeg mask of dst face        	-> faces from dst faceset should be labeled
XSeg-prd*XSeg-dst - the smallest area of both

if workspace\model folder contains trained XSeg model, then merger will use it,
otherwise you will get transparent mask by using XSeg-* modes.

Some screenshots:
XSegEditor: https://i.imgur.com/7Bk4RRV.jpg
trainer   : https://i.imgur.com/NM1Kn3s.jpg
merger    : https://i.imgur.com/glUzFQ8.jpg

example of the fake using 13 segmented dst faces
          : https://i.imgur.com/wmvyizU.gifv
  • Loading branch information
iperov committed Mar 24, 2020
1 parent eddebed commit 01d81674fdf0acb1b896db3b984828c9812d9111
Showing with 2,009 additions and 24 deletions.
  1. +2 −2 README.md
  2. +10 −0 XSegEditor/QCursorDB.py
  3. +21 −0 XSegEditor/QIconDB.py
  4. +72 −0 XSegEditor/QStringDB.py
  5. +1,265 −0 XSegEditor/XSegEditor.py
  6. BIN XSegEditor/gfx/cursors/cross_blue.png
  7. BIN XSegEditor/gfx/cursors/cross_green.png
  8. BIN XSegEditor/gfx/cursors/cross_red.png
  9. BIN XSegEditor/gfx/fonts/NotoSans-Medium.ttf
  10. BIN XSegEditor/gfx/icons/app_icon.png
  11. BIN XSegEditor/gfx/icons/delete_poly.png
  12. BIN XSegEditor/gfx/icons/down.png
  13. BIN XSegEditor/gfx/icons/left.png
  14. BIN XSegEditor/gfx/icons/poly_color.psd
  15. BIN XSegEditor/gfx/icons/poly_color_blue.png
  16. BIN XSegEditor/gfx/icons/poly_color_green.png
  17. BIN XSegEditor/gfx/icons/poly_color_red.png
  18. BIN XSegEditor/gfx/icons/poly_type_exclude.png
  19. BIN XSegEditor/gfx/icons/poly_type_include.png
  20. BIN XSegEditor/gfx/icons/poly_type_source.psd
  21. BIN XSegEditor/gfx/icons/pt_edit_mode.png
  22. BIN XSegEditor/gfx/icons/pt_edit_mode_source.psd
  23. BIN XSegEditor/gfx/icons/redo_pt.png
  24. BIN XSegEditor/gfx/icons/redo_pt_source.psd
  25. BIN XSegEditor/gfx/icons/right.png
  26. BIN XSegEditor/gfx/icons/undo_pt.png
  27. BIN XSegEditor/gfx/icons/undo_pt_source.psd
  28. BIN XSegEditor/gfx/icons/up.png
  29. BIN XSegEditor/gfx/icons/view_baked.png
  30. +152 −0 core/imagelib/SegIEPolys.py
  31. +1 −0 core/imagelib/__init__.py
  32. +262 −0 core/qtex/QSubprocessor.py
  33. +83 −0 core/qtex/QXIconButton.py
  34. +34 −0 core/qtex/QXMainWindow.py
  35. +3 −0 core/qtex/__init__.py
  36. +79 −0 core/qtex/qtex.py
  37. +1 −1 localization/__init__.py
  38. +12 −0 localization/localization.py
  39. +6 −16 main.py
  40. +2 −1 requirements-cuda.txt
  41. +2 −2 samplelib/Sample.py
  42. +2 −2 samplelib/SampleGeneratorFaceXSeg.py
@@ -121,9 +121,9 @@ Unfortunately, there is no "make everything ok" button in DeepFaceLab. You shoul
||bitcoin:bc1qkhh7h0gwwhxgg6h6gpllfgstkd645fefrd5s6z|
|Alipay 捐款|![](doc/Alipay_donation.jpg)|
|||
|Last donations|10$ ( 朱 阳阳 )|
|Last donations|50$ ( Tomas Hajka )|
||10$ ( 朱 阳阳 )|
||24$ ( NextFace )|
||10$ ( Amien Phillips )|
|||
|Collect facesets|You can collect faceset of any celebrity that can be used in DeepFaceLab and share it [in the community](https://mrdeepfakes.com/forums/forum-celebrity-facesets)|
|||
@@ -0,0 +1,10 @@
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *

class QCursorDB():
@staticmethod
def initialize(cursor_path):
QCursorDB.cross_red = QCursor ( QPixmap ( str(cursor_path / 'cross_red.png') ) )
QCursorDB.cross_green = QCursor ( QPixmap ( str(cursor_path / 'cross_green.png') ) )
QCursorDB.cross_blue = QCursor ( QPixmap ( str(cursor_path / 'cross_blue.png') ) )
@@ -0,0 +1,21 @@
from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *


class QIconDB():
@staticmethod
def initialize(icon_path):
QIconDB.app_icon = QIcon ( str(icon_path / 'app_icon.png') )
QIconDB.delete_poly = QIcon ( str(icon_path / 'delete_poly.png') )
QIconDB.undo_pt = QIcon ( str(icon_path / 'undo_pt.png') )
QIconDB.redo_pt = QIcon ( str(icon_path / 'redo_pt.png') )
QIconDB.poly_color_red = QIcon ( str(icon_path / 'poly_color_red.png') )
QIconDB.poly_color_green = QIcon ( str(icon_path / 'poly_color_green.png') )
QIconDB.poly_color_blue = QIcon ( str(icon_path / 'poly_color_blue.png') )
QIconDB.poly_type_include = QIcon ( str(icon_path / 'poly_type_include.png') )
QIconDB.poly_type_exclude = QIcon ( str(icon_path / 'poly_type_exclude.png') )
QIconDB.left = QIcon ( str(icon_path / 'left.png') )
QIconDB.right = QIcon ( str(icon_path / 'right.png') )
QIconDB.pt_edit_mode = QIcon ( str(icon_path / 'pt_edit_mode.png') )
QIconDB.view_baked = QIcon ( str(icon_path / 'view_baked.png') )
@@ -0,0 +1,72 @@
from localization import system_language


class QStringDB():

@staticmethod
def initialize():
lang = system_language

if lang not in ['en','ru','zn']:
lang = 'en'

QStringDB.btn_poly_color_red_tip = { 'en' : 'Poly color scheme red',
'ru' : 'Красная цветовая схема полигонов',
'zn' : '多边形配色方案红色',
}[lang]

QStringDB.btn_poly_color_green_tip = { 'en' : 'Poly color scheme green',
'ru' : 'Зелёная цветовая схема полигонов',
'zn' : '多边形配色方案绿色',
}[lang]

QStringDB.btn_poly_color_blue_tip = { 'en' : 'Poly color scheme blue',
'ru' : 'Синяя цветовая схема полигонов',
'zn' : '多边形配色方案蓝色',
}[lang]

QStringDB.btn_view_baked_mask_tip = { 'en' : 'View baked mask',
'ru' : 'Посмотреть запечёную маску',
'zn' : '查看遮罩通道',
}[lang]

QStringDB.btn_poly_type_include_tip = { 'en' : 'Poly include mode',
'ru' : 'Режим полигонов - включение',
'zn' : '多边形包含模式',
}[lang]

QStringDB.btn_poly_type_exclude_tip = { 'en' : 'Poly exclude mode',
'ru' : 'Режим полигонов - исключение',
'zn' : '多边形排除方式',
}[lang]

QStringDB.btn_undo_pt_tip = { 'en' : 'Undo point',
'ru' : 'Отменить точку',
'zn' : '撤消点',
}[lang]

QStringDB.btn_redo_pt_tip = { 'en' : 'Redo point',
'ru' : 'Повторить точку',
'zn' : '重做点',
}[lang]

QStringDB.btn_delete_poly_tip = { 'en' : 'Delete poly',
'ru' : 'Удалить полигон',
'zn' : '删除多边形',
}[lang]

QStringDB.btn_pt_edit_mode_tip = { 'en' : 'Edit point mode ( HOLD CTRL )',
'ru' : 'Режим правки точек',
'zn' : '编辑点模式 ( 按住CTRL )',
}[lang]

QStringDB.btn_prev_image_tip = { 'en' : 'Save and Prev image\nHold SHIFT : accelerate\nHold CTRL : skip non masked\n',
'ru' : 'Сохранить и предыдущее изображение\nУдерживать SHIFT : ускорить\nУдерживать CTRL : пропустить неразмеченные\n',
'zn' : '保存和上一张图片\n按住SHIFT : 加快\n按住CTRL : 跳过未标记的\n',
}[lang]
QStringDB.btn_next_image_tip = { 'en' : 'Save and Next image\nHold SHIFT : accelerate\nHold CTRL : skip non masked\n',
'ru' : 'Сохранить и следующее изображение\nУдерживать SHIFT : ускорить\nУдерживать CTRL : пропустить неразмеченные\n',
'zn' : '保存并下一张图片\n按住SHIFT : 加快\n按住CTRL : 跳过未标记的\n',
}[lang]


0 comments on commit 01d8167

Please sign in to comment.
You can’t perform that action at this time.