Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print out which image caused error #147

Merged
merged 1 commit into from Feb 8, 2018
Merged

Conversation

leijurv
Copy link
Contributor

@leijurv leijurv commented Feb 8, 2018

Exception message on encountering a corrupted image used to be:

libpng error: IDAT: CRC error.04081, loss_B: 0.05761
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/home/leijurv/Documents/faceswap/lib/utils.py", line 42, in run
    for item in self.generator:
  File "/home/leijurv/Documents/faceswap/lib/training_data.py", line 43, in minibatch
    rtn = numpy.float32([read_image(data[j]) for j in range(i,i+size)])
  File "/home/leijurv/Documents/faceswap/lib/training_data.py", line 43, in <listcomp>
    rtn = numpy.float32([read_image(data[j]) for j in range(i,i+size)])
  File "/home/leijurv/Documents/faceswap/lib/training_data.py", line 25, in read_image
    image = cv2.imread(fn) / 255.0
TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'

Now, it prints out which image specifically caused the error:

libpng error: IDAT: CRC error.03255, loss_B: 0.04847
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/home/leijurv/Documents/faceswap/lib/training_data.py", line 26, in read_image
    image = cv2.imread(fn) / 255.0
TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/home/leijurv/Documents/faceswap/lib/utils.py", line 42, in run
    for item in self.generator:
  File "/home/leijurv/Documents/faceswap/lib/training_data.py", line 46, in minibatch
    rtn = numpy.float32([read_image(data[j]) for j in range(i,i+size)])
  File "/home/leijurv/Documents/faceswap/lib/training_data.py", line 46, in <listcomp>
    rtn = numpy.float32([read_image(data[j]) for j in range(i,i+size)])
  File "/home/leijurv/Documents/faceswap/lib/training_data.py", line 28, in read_image
    raise Exception("Error while reading image",fn)
Exception: ('Error while reading image', '/home/leijurv/faceswap/data/B/batch5_3970.png')

@Clorr Clorr merged commit 20753a6 into deepfakes:master Feb 8, 2018
@Clorr
Copy link
Contributor

Clorr commented Feb 8, 2018

Thanks for the commit ;-)

leebrian added a commit to leebrian/faceswap that referenced this pull request Apr 3, 2018
* Adds arg to select trainer used to create model (deepfakes#105)

Stops the layer count mismatch when a LowMem model is converted using the Original model.

* Added CUDA link for Ubuntu

* Adds information about running scripts with help (deepfakes#111)

Updated information so that users can quickly see the available parameters for the various scripts.
Also made the section headers correspond with the actual scripts - e.g. extract, train, convert

* Fix breakage on some versions of python3 (on Ubuntu 17.10 for sure)

Also gives better info when you don't include any arguments (or bad ones)

* Add demosaic (deepfakes#131)

Copy from https://github.com/dfaker/df/blob/master/merge_faces_larger.py#L127-L133

* Update Convert_Masked.py (deepfakes#130)

correct issue when seamless_clone is true

* Misc updates on master before GAN. Added multithreading + mmod face detector (deepfakes#109)

* Preparing GAN plugin

* Adding multithreading for extract

* Adding support for mmod human face detector

* Adding face filter argument

* Added process number argument to multiprocessing extractor.

Fixed progressbar for multiprocessing.

* Added tiff as image type.
compression artefacts hurt my feelings.

* Cleanup

* Changes I forgot to push :-/ (deepfakes#136)

* Adding GAN plugin (deepfakes#102)

Update GAN plugin to latest official version

* Correcting model paths

* Fixing ConvertImage has no attribute check_skip

deepfakes#143

* print out which image caused error (deepfakes#147)

* Added serializers.

This speeds up convert speed x4 on my machine.

* lowercase mask_type (deepfakes#181)

The mask_type passed in to this function is lowercase, changing string literals to match.

* New arg. (deepfakes#177)

* Documentation and grammar (deepfakes#167)

* Fixed gramatical mistake

* Improved documentation and fixed spelling and grammar

* fixes for some comments

* should fix full paths, and windowspath issues

* didnt import os in extract.

* fixed and tested, briefly with gan and original.

* Skip images that arent in the alignments.json

* no os import in convert.

* fix for have_face.

* Update INSTALL.md (deepfakes#178)

INSTALL.md now aligns with requirements in script.

* Clearer requirements for each platform (deepfakes#183)

* Adding allow_growth option (deepfakes#140)

* Add GNU General Public License v3.0

* Refactoring scripts/extract.py (deepfakes#216)

Refactored extract to have handleImage take in a image instead of a filename. This will be useful for writing future extensions for extract where you may not have a file on disk but you do have a image in memory. For example extracting faces from a video.

* Match the command to split/generate video (deepfakes#196)

* Correcting a bug in handleImage call

* Correcting extract.py

* port 'face_alignment' from PyTorch to Keras. (deepfakes#228)

* port 'face_alignment' from PyTorch to Keras. It works x2 faster, but initialization takes 20secs.

2DFAN-4.h5 and mmod_human_face_detector.dat included in lib\FaceLandmarksExtractor

fixed dlib vs tensorflow conflict: dlib must do op first, then load keras model, otherwise CUDA OOM error

if face location not found by CNN, its try to find by HOG.

removed this:
-        if face.landmarks == None:
-            print("Warning! landmarks not found. Switching to crop!")
-            return cv2.resize(face.image, (size, size))
because DetectedFace always has landmarks

* removing DetectedFace.landmarks

* fix issue deepfakes#234 (deepfakes#235)

* fix deepfakes#233, deepfakes#237, + working with any batch size (deepfakes#236)

* fix working with any batch size

* fix deepfakes#233 - primordial bug when dlib recognizes less faces than FakeApp, reason: rgb-bgr order affects chance of face recognition. Numpy.imread and cv2.imread loads same image in different rgb-bgr order.

fix deepfakes#237

* fix

* Allows applying dilation by passing negative erosion kernel values. If value is negative, … (deepfakes#238)

* Allows for negative erosion kernel for -e arg. If value is negative, it turns it into a dilation kernel, which allow facehullandrect to cover more space. Can help to cover double eyebrows. Also could be useful with Masked converter for GAN that oatsss is working on.

* Update convert.py

Modified argument help to clarify the effects of erosion and dilation as parameters

* Add debugging option for drawing landmarks on face extraction. (deepfakes#199)

* backslash problem correction

fix for deepfakes#239

* Skip already extracted frames when using extract.py (deepfakes#214)

* Pytorch and face-alignment

* Skip processed frames when extracting faces.

* Reset to master version

* Reset to master

* Added --skip-existing argument to Extract script. Default is to NOT skip already processed frames.
Added logic to write_alignments to append new alignments (and preserve existing ones)
to existing alignments file when the skip-existing option is used.

* Fixed exception for --skip-existing when using the convert script

* Sync with upstream

* Fixed error when using Convert script.

* Bug fix

* Merges alignments only if --skip-existing is used.

* Creates output dir when not found, even when using --skip-existing.

* Update GAN64 to v2 (deepfakes#217)

* Clearer requirements for each platform

* Refactoring of old plugins (Model_Original + Extract_Align) + Cleanups

* Adding GAN128

* Update GAN to v2

* Create instance_normalization.py

* Fix decoder output

* Revert "Fix decoder output"

This reverts commit 3a8ecb8.

* Fix convert

* Enable all options except perceptual_loss by default

* Disable instance norm

* Update Model.py

* Update Trainer.py

* Match GAN128 to shaoanlu's latest v2

* Add first_order to GAN128

* Disable `use_perceptual_loss`

* Fix call to `self.first_order`

* Switch to average loss in output

* Constrain average to last 100 iterations

* Fix math, constrain average to intervals of 100

* Fix math averaging again

* Remove math and simplify this damn averagin

* Add gan128 conversion

* Update convert.py

* Use non-warped images in masked preview

* Add K.set_learning_phase(1) to gan64

* Add K.set_learning_phase(1) to gan128

* Add missing keras import

* Use non-warped images in masked preview for gan128

* Exclude deleted faces from conversion

* --input-aligned-dir defaults to "{input_dir}/aligned"

* Simplify map operation

* port 'face_alignment' from PyTorch to Keras. It works x2 faster, but initialization takes 20secs.

2DFAN-4.h5 and mmod_human_face_detector.dat included in lib\FaceLandmarksExtractor

fixed dlib vs tensorflow conflict: dlib must do op first, then load keras model, otherwise CUDA OOM error

if face location not found by CNN, its try to find by HOG.

removed this:
-        if face.landmarks == None:
-            print("Warning! landmarks not found. Switching to crop!")
-            return cv2.resize(face.image, (size, size))
because DetectedFace always has landmarks

* Enabled masked converter for GAN models

* Histogram matching, cli option for perceptual loss

* Fix init() positional args error

* Add backwards compatibility for aligned filenames

* Fix masked converter

* Remove GAN converters

* Fix line endings (deepfakes#266)

* Remove files with line-ending issues

* Add back files with line-ending issues

* Fixes (deepfakes#267)

* Use k-nn for face filtering (deepfakes#262)

* Add negative filters for face detection

When detecting faces that are very similar, the face recognition can
produce positive results for similar looking people. This commit allows
the user to add multiple positive and negative reference images. The
facedetection then calculates the distance to each reference image
and tries to guess which is more likely using the k-nearest method.

* Do not calculate knn if no negative images are given

* Clean up outputting

* sorttool.py

* PluginLoader.get_available_models()
PluginLoader.get_default_model()
provides easy integration of Model_* folders without changing convert.py and train.py

* Fixing number of args

* Add image rotation for detecting more faces and dealing with awkward angles (deepfakes#253)

* Image rotator for extract and convert ready for testing

* Revert "Image rotator for extract and convert ready for testing"

This reverts commit bbeb19e.

Error in extract code

* add image rotation support to detect more faces

* Update convert.py

Amended to do a single check for for rotation rather than checking twice. Performance gain is likely to be marginal to non-existent, but can't hurt.

* Update convert.py

remove type

* cli.py: Only output message on verbose. Convert.py: Only check for rotation amount once

* Changed command line flag to take arguments to ease future development

* Realigning for upstream/Master

* Minor fix

* Fix for missing default rotation value (deepfakes#269)

* Image rotator for extract and convert ready for testing

* Revert "Image rotator for extract and convert ready for testing"

This reverts commit bbeb19e.

Error in extract code

* add image rotation support to detect more faces

* Update convert.py

Amended to do a single check for for rotation rather than checking twice. Performance gain is likely to be marginal to non-existent, but can't hurt.

* Update convert.py

remove type

* cli.py: Only output message on verbose. Convert.py: Only check for rotation amount once

* Changed command line flag to take arguments to ease future development

* Realigning for upstream/Master

* Minor fix

* Change default rotation value from None to 0

* Add Improved AutoEncoder model. (deepfakes#251)

* Add Improved AutoEncoder model.

* Refactoring Model_IAE to match the new model folder structure

* Add Model_IAE in plugins

* Improving performance of extraction. Two main changes to improve the … (deepfakes#259)

* Improving performance of extraction. Two main changes to improve the most recent modifications to extract: 1st FaceLandmarkExtractor would try to use cnn first, then try hog. The problem was that this reduced the speed by 4 for images where cnn didn't find anything, and most of the times hog wouldn't find anything either or it would be a bad extract. For me it wasn't worth it. With this you can specify on input -D if you want to use hog, cnn, or all. 'all' will try cnn, then hog like FaceLandmarkExtractor was doing. cnn or hog will just use 1 detection method. 2nd change is a rehaul of the verbose parameter. Now warnings when a face is not detected will just be shown if indicated by -v or --verbose. This restores the verbose function to what it once was. With this change I was able to process 1,000 per each 4 minutes regardless if faces were detected or not. Performance improvement just applies to not detected images but I normally will have lots of images without clear faces in my set, so I figured it would impact others. Also the introduction of 'all' would allow trying other models together more easily in the future.

* Update faces_detect.py

* Update extract.py

* Update FaceLandmarksExtractor.py

* spacing fix

* Renaming model files

* Fix to Model_IAE not working (deepfakes#275)

Fix to Model_IAE giving excess positional arguments error.

* -by face

* Add Multi-GPU support (deepfakes#272)

* Add Improved AutoEncoder model.

* Refactoring Model_IAE to match the new model folder structure

* Add Model_IAE in plugins

* Add Multi-GPU support

I added multi-GPU support to the new model layout.  Currently, Original is not tested (due to OOM on my 2x 4gb 970s).  LowMem is not tested with the current commit due to it not being available since the new pluginloader misses it.

* Fix broken multigpu GAN loading. (deepfakes#280)

* Fix broken multigpu GAN loading.

Fix for loading into the multi GPU model when it needs to load/save the original model.

* Move reference change.

Move reference change for single model so it is defined before load

* Add mutliGPU support to GAN128

Added support for mutligpu to GAN128

* Added command line used in other relevant steps

* Output Sharpening Added (deepfakes#285)

* Updated to support Output Sharpening arguments

Two new types of output sharpening methods have been added.

One that deals with a Box Blur method and the other with a Gaussian Blur method.

Box Blur method can be called using argument '-sh bsharpen' --- This method is not dynamic and can produce strong sharpening on your images. Sometimes it can yield great results and sometimes entirely the opposite.

Gaussian Blur method can be called using argument '-sh gsharpen' --- This method is dynamic and tries to adjust to your data set. As a result, while the sharpening effect might not be as strong as bsharpen, it is bound to produce a more natural looking sharpened image.

By default the parameter is set to none which will not run any sharpening on your output.

* Output Sharpening added

Two ways of sharpening your output have been added

-sh bsharpen
-sh gsharpen

* Update convert.py

* Fix padding problem with gan conversion (deepfakes#289)

* Fix padding problem with gan conversion

* Revert gan transform_args

* Align Eyes Horizontally After Umeyama + Blur Detection (deepfakes#242)

* Align eyes after umeyama

* Remove comment

* Add cli option

* Update Extract_Crop.py

* Fix convert

* Add blur threshold

* Use mask in blur detection

* Improve blur detection

* Fix indents

* Update extract.py

* Converted LowMem model to the new structure (deepfakes#292)

* converted lowmem to the new structure

* removed old lowmem

* Fix gan128 (deepfakes#288)

* Complete parity fix for GAN128.

This brings GAN128 to parity with GAN in terms of multi GPU support.

Unfortunately, it fails to run due to a naming error.

('The name "model_4" is used 2 times in the model. All layer names should be unique. Layer names: ', ['input_6', 'lambda_1', 'lambda_2', 'model_6', 'model_4', 'model_4'])

* Fix for GAN128 until deepfakes#287 can be resolved

Issue deepfakes#287 details why GAN128 cannot be fixed until Keras is fixed upstream.

* Changes to PR as per Clorr

Made changes to error handling to split into separate PR as requested by Clorr.

* Fixed error handling in train.py (deepfakes#293)

Fixed the error handling in train.py so it doesn't swallow tracelogs.

* Adding tools.py as main script for using tools, as well as integrating all feature requests from deepfakes#255 and deepfakes#278 (deepfakes#298)

* Add tools.py command and control script for use as the main interface for various tools. The structure and approach is the same as faceswap.py
Add many new features to tools/sort.py: various new sorting methods, grouping by folders, logging file renaming/movemeng, keeping original files in the input directory and improved cli options documentation. Argument parsing has been re-written to inteface with tools.py
Add __init__.py empty file in tools directory for python to register it as a module so that sort.py and future tools can be easily imported.

* Fix various bugs where the correct sorting method would not get called.
Add new sorting methon: face-cnn-dissim.
Update help documentation for face-cnn-dissim.
Change default grouping to rename.
Update initial print in all sorting/grouping methods to say precisely which method is being used.

* Major refactor and redesign.
Use dynamic method allocation to avoid large amounts of nested if-elif statements in process() function and to allow easily combine sort and group methods.

Change cli arguments to make them more intuitive and work with the new design.
Previous: '-g/--grouping' -> '-f/--final-processing' {folders,rename}
Previous: '-by/--by' -> '-s/--sort-by' {blur,face,face-cnn,face-cnn-dissim,face-dissim,hist,hist-dissim}
New: '-g/--group-by' {blur,face,face-cnn,hist}
Add: '--logfile' -> '-lg/--logfile' PATH_TO_LOGFILE

Greatly improve grouping performance.
Grouping now has to sort using one of the sorting methods which makes the grouping stable and no longer dependent on how well the the target files are already sorted.
Sorting and grouping methods can be combined in any way. If no -g/--group-by is specified by user, it will default to group by the non '-dissim' version of sort method.
Different combinations of sorting and grouping methods work well for different sets of data.

Fixes
Fix progress updates not showing properly by setting them to print to stdout instead of stderror.
Fix bug in grouping by face-cnn where wrong score method was being called.

Misc
Add documentation for reload_list() and splice_lists() methods because it's not obvious what they do.
Add warning message to tools.py to tell users to make sure they understand how the tool they want to use works before using it.
Add warning message to tools/sort.py to tell users to make sure they undrestand how the sort tool works before using it.
Update help documentation to reflect new functionality and options.
Set defaults for group by face-cnn to work properly with the correct score method.
Amend commit in order to sign it.

* Perform unittests for all options and combinations of sort and group methods: everything OK.
Fix typos in help documentation.

* Mask refinement option for GAN128 (deepfakes#308)

I was experimenting on GAN128 and saw that mask refinement option was missing so i added that.
Its default value is False and its optional after 15k iterations.

Based on this: https://render.githubusercontent.com/view/ipynb?commit=87d6e7a28ce754acd38d885367b6ceb0be92ec54&enc_url=68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f7368616f616e6c752f66616365737761702d47414e2f383764366537613238636537353461636433386438383533363762366365623062653932656335342f46616365537761705f47414e5f76325f737a3132385f747261696e2e6970796e62&nwo=shaoanlu%2Ffaceswap-GAN&path=FaceSwap_GAN_v2_sz128_train.ipynb&repository_id=115182783&repository_type=Repository#Tips-for-mask-refinement-(optional-after-%3E15k-iters)

* Fix "raw"/"masked" preview labeling

* Fix "raw"/"masked" labeling

* sorttool - fixes, and added --sort-by face-yaw (deepfakes#312)

* Renaming BGR/RGB inputs

* Fix to an UnboundLocalError due to rename. (deepfakes#318)

* FaceLandmarksExtractor comment (deepfakes#317)

* Adds support for arbitrary image rotations (deepfakes#309)

* Add support for user-specified rotation angle in extract

* Added rotation-angle-list option to enumerate a list of angles to rotate through

* Adjust rotation matrix translation coords to avoid cropping

* Merged rotation-angle and rotation-angle-list options into rotate_images option

* Backwards compatibility

* Updated check whether to run image rotator

* Switched rotation convention to use positive angle = clockwise rotation, for backwards compatibility

* Revert " Fix to an UnboundLocalError due to rename." -- Bind the variable in question, rather than replace it with another parameter (deepfakes#320)

* Switching naming of _bgr as discussed to reverse detector call

Switching naming of _bgr as discussed to reverse detector call
renaming iterator in loop for clarity

* Revert "Adds support for arbitrary image rotations (deepfakes#309)"

This reverts commit 44dfd9d.

* Revert "FaceLandmarksExtractor comment (deepfakes#317)"

This reverts commit f79c487.

* Revert "Fix to an UnboundLocalError due to rename. (deepfakes#318)"

This reverts commit 2e2dc84.

* Correction to UnboundLocalError

Tested that "sort -s face-cnn" works correctly after chagnes to FaceLandmarksExtractor.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants