-
Notifications
You must be signed in to change notification settings - Fork 137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fail-fast or skipping of problematic images #15
Comments
Thanks for reaching out on this, and sorry for your troubles @tokee. I agree that failing fast is nice, though skipping the problematic images could work as well. I'm curious about the corrupt images--were the inputs corrupt, or the outputs, or both? Could you send a few example images that produced corrupt outputs? If so, those samples could help determine how best to treat some cases... |
It was definitely the input images that were corrupt. I could dig up some samples, but as ImageMagic's own |
@tokee I just pushed up a branch |
Looks good! It catches both parentheses and spaces in filenames. Thanks! |
Awesome, thanks for following up! |
I am in the process of testing pix-plot with 26K images with varying names.
Some of them contained parentheses, which caused pix-plot to fail very late in the process. Others had an apostrophe, which gave the same error.
I checked and the file
output/thumbs/32px/bi_20150530-1609_2_AstaCykelSt'tteben.jpg
does exist, but I can understand why it could give problems in a script.Some images were corrupt, which (guessing here) meant that the thumbnail-processing left no file, which again caused some later step to fail with a different error. Unfortunately I did not keep that error message.
So far I have handled the problem by finding & removing files with problematic characters when the script throws an exception and removing all corrupt images using
for F in sources_sshfs/*.jpg; do if [[ -z $(identify "$F" 2> /dev/null) ]]; then echo "Removing $F" ; rm "$F" ; fi ; done
, but I haven't completed a full run of the 26K images yet.Whether due to naming or format problems, the overall problem is the same: The script needs to run for 10+ minutes before it fails. Preferably it could be made to just skip the problematic images, but alternatively it would be nice if it could fail early in the process?
The text was updated successfully, but these errors were encountered: