Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude avatar sizes 16x16, 24x24, 30x30, 40x40, 48x48 and 64x64 #32

Merged
merged 2 commits into from
Dec 12, 2018

Conversation

Fusl
Copy link
Member

@Fusl Fusl commented Dec 12, 2018

21:27 <Fusl> so looking at https://github.com/ArchiveTeam/tumblr-grab/blob/master/tumblr.lua#L52 i can see that we're excluding 16x16 and 64x64 sized avatars, but what size do we actually want to exclude because i can see a wide variety of different sizes that we may want to include as well that take half of my crawlers time
21:27 <Fusl> 16, 24, 30, 40, 48, 64, 96, 128, 512
21:30 <Fusl> different sizes of avatars i'm so far downloading according to wget.log: http://xor.meo.ws/YNf9yTudvJCd3pBOIyw52PDK8RPwYCP4.txt
21:30 <Fusl> as one can see, 30x30 avatars are VERY common
21:30 <Fusl> as are 24x24 and 40x40
21:31 <Fusl> i'm going to open a PR to exclude everything below 64x64
21:31 <@kiska> Next update: Remove all avatars < 100 x 100 px

21:27 <Fusl> so looking at https://github.com/ArchiveTeam/tumblr-grab/blob/master/tumblr.lua#L52 i can see that we're excluding 16x16 and 64x64 sized avatars, but what size do we actually want to exclude because i can see a wide variety of different sizes that we may want to include as well that take half of my crawlers time
21:27 <Fusl> 16, 24, 30, 40, 48, 64, 96, 128, 512
21:30 <Fusl> different sizes of avatars i'm so far downloading according to wget.log: http://xor.meo.ws/YNf9yTudvJCd3pBOIyw52PDK8RPwYCP4.txt
21:30 <Fusl> as one can see, 30x30 avatars are VERY common
21:30 <Fusl> as are 24x24 and 40x40
21:31 <Fusl> i'm going to open a PR to exclude everything below 64x64
21:31 <@kiska> Next update: Remove all avatars < 100 x 100 px
@Arkiver2 Arkiver2 merged commit dd22eeb into ArchiveTeam:master Dec 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants