-
Notifications
You must be signed in to change notification settings - Fork 210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
about Data preprocess #9
Comments
For the data preprocessing, please check section 4.1 in the paper. Feel free to comment if you have any questions. |
|
|
Time-incosistency between generated frames is a common issue in Image-to-Image translation methods, especially when it is applied to generate sequence frames. The keypoint here is that: Semantic ambiguity exists in the training dataset. To alleviate this phenomenon, one should try to keep the training set semantic consistency. For example, you may need to carefully cut and crop the face region and make it consistent for all frames, smooth the detected facial landmarks to reduce the temporal jitters, add more conditions on frequent-changed regions. Also, you can apply a time-conditional schedule, e.g., feed the history generated frames to the renderer as an additional condition. Anyway, you should carefully check your dataset and design your network schedule because it is the ambiguity that cause the issue. Editing the feature maps far away from the training span will lead to artifacts - that is for sure a common issue for any learning method. You must edit the feature maps using samples in the training span or it will generate artifacts. Actually, calling it as an issue is not appropriate, because it is reasonable that the network should only work well when input in the training span, right? After all, networks can't imagine. If you want to edit the feature map in a large scale, the thorough solution is to train generalized models using a large enough dataset, and that is a much harder problem. Anyway, you must set some rules on the models, for example, at least the feature maps should look like humans -- that is models will fail when you edit the landmarks like a dog. Any model has its borders, just like a saying goes, rubbish in, rubbish out. |
Impressive job!
I wonder how to preprocess the images. Specifically, could you please share the scripts on choosing the four candidate images from the sequences and how to draw the shoulder edges since the landmark detectors I have found are all face landmark detectors.
Thanks !
The text was updated successfully, but these errors were encountered: