Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken Image Reporting #3610

Closed
BrokenEagle opened this issue Apr 7, 2018 · 14 comments
Closed

Broken Image Reporting #3610

BrokenEagle opened this issue Apr 7, 2018 · 14 comments
Labels

Comments

@BrokenEagle
Copy link
Collaborator

I figured I'd open an issue to track this.

Range checked: 0..711145
Broken thumbnails: 30
Broken samples: 1
Broken originals: 1
Broken dictionary: checkallimage1.txt

The above JSON file has 3 keys, "previewsize" (thumbnail), "samplesize" (850px version), "fullsize" (original version). The keys under each of those is the post # and their value indicates which HTTP error code was returned.

@r888888888
Copy link
Collaborator

The base_dir parameter for the SFTP storage managers wasn't configured so replacements on the archives servers wasn't working. That should be fixed now. I've manually synced the files in 1.txt.

@BrokenEagle
Copy link
Collaborator Author

Range checked: 711145..1027186
Broken thumbnails: 0
Broken samples: 11
Broken originals: 11
Broken dictionary: checkallimage2.txt

Same format as before. The broken samples and originals happen to all be from the same posts.

@r888888888
Copy link
Collaborator

2 has been synced.

@BrokenEagle
Copy link
Collaborator Author

Range checked: 1027186..1465992
Broken thumbnails: 0
Broken samples: 61
Broken originals: 61
Broken dictionary: checkallimage.3.txt

Same format as before. Like last time, the broken samples and originals happen to all be from the same posts.

@r888888888
Copy link
Collaborator

3 has been synced.

@BrokenEagle
Copy link
Collaborator Author

Range checked: 1465992..2293992
Broken thumbnails: 0
Broken samples: 527
Broken originals: 526
Broken dictionary: checkallimage.4.txt

Unlike the previous times, there is actually a slight difference between the broken samples and originals. In this case, post 1486035 has a broken sample image, but not a broken full image. Besides that, all other broken samples and originals happen to all be from the same posts.

@r888888888
Copy link
Collaborator

4 has been synced.

@BrokenEagle
Copy link
Collaborator Author

Range checked: 2293992..3196042
Broken thumbnails: 7
Broken samples: 8
Broken originals: 8
Broken dictionary: checkallimage.5.txt

So not as many as last time, however this time none of the image size sets match with each other, as there are 1 or 2 post differences.

@r888888888
Copy link
Collaborator

2737344, 2770039 can't be found. The others were fixed.

@BrokenEagle
Copy link
Collaborator Author

Range checked: 3196042...3260255
Broken thumbnails: 6
Broken samples: 7
Broken originals: 6
Broken dictionary: CheckAllImagesourcefileAsync.txt

Most of the broken images are the same as before.

2737344, 2770039 can't be found. The others were fixed.

Many/most of the broken images can be found at the other boorus. For the above, I just used the md5 of the original image on Danbooru plus the md5 metatag to find the above image.

@BrokenEagle
Copy link
Collaborator Author

Since #3954 I've been scanning all images from the beginning again, and just finally caught up to the latest posts.

Range checked: 0..3338191
Broken thumbnails:7
Broken samples: 63
Broken originals: 92
Broken dictionary: checkallimagejson.7.txt

Same JSON format as before: previewsize, samplesize, fullsize. Each key underneath those is the post # and the HTTP error code, which all happen to be 404.

@stale
Copy link

stale bot commented Jul 29, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jul 29, 2019
@evazion evazion added Bug and removed stale labels Jul 30, 2019
@BrokenEagle
Copy link
Collaborator Author

I did another recent scan and also did a visual confirmation of the missing images.
Broken thumbnails: 6 (favgroup #2491)
Broken samples: 231 (favgroup #2573)
Broken originals: 8 (favgroup #1465)

The ones that still have working originals should be easy to fix, but the others may require downloading if the file still exists. If not, then those posts should be permanently deleted.

@nonamethanks
Copy link
Member

I did a scan a while ago and fixed all the fixable ones, and replaced corrupted ones with images from gelbooru when it was possible. Closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants