Thumbnail support for regular pixiv image URLs #118

Open
wants to merge 1 commit into
from

Conversation

Projects
None yet
3 participants
Contributor

Kovensky commented Aug 1, 2012

Regular pixiv URLs don't have enough information in the URL to derive the
image's URL, so we have to do some HTML parsing. Since we can't block or the UI
freezes, we do the request as part of ImageSizeCheckClient's check.

Pixiv URLs also need a bit of special treatment. Clicking the thumbnail would
open the direct link to the image in the browser, which isn't very useful because
you would get a 403 back (or a small image), so we inject a link to the web
interface for the image.

This (apparently) passes Instruments' Zombie check and Leak check;
slowly learning the Obj-C memory management stuff...

Contributor

Kovensky commented Aug 1, 2012

I think it's more of a case of simply not having an API to get the thumbnails. Ideally you should be able to get a thumbnail from just the URL, but it's missing a few of the path variables, so we have to scrape to get the proper path. I've talked with a few pixiv people and they didn't seem to mind as long as I kept to small size images (such as /dic/320x320/), though they weren't the backend people.

d4rkie commented Aug 1, 2012

unfortunately, we don't have a proper API yet, so scraping is the only way. I will consult with the team if we can make a simple API for thumbnails later today

Owner

psychs commented Aug 1, 2012

I see the situation. I have experience on some scraping codes, but it broke very easily when the server changed HTML structure.

Thanks d4rkie. If they would want to contact me directly, tell them to send me a mail without any hesitations. I would be happy to support Pixiv thumbnails in LimeChat.

d4rkie commented Aug 2, 2012

talked with the team. you can use the Open Graph API we have for Facebook. we will have to whitelist the LimeChat user-agent, though, so please provide one.

Owner

psychs commented Aug 2, 2012

Cool. Thanks! User agent will be like "LimeChat/2.31".

Contributor

Kovensky commented Aug 2, 2012

Hmm, with the OpenGraph API we could actually provide thumbnails for more things than just Pixiv. Should I refactor this into a generic OpenGraph client and add a whitelist of sites that implement OpenGraph? It'll still need a bit of pixiv-specific code, but a lot less than now.

Owner

psychs commented Aug 2, 2012

I think we just need Pixiv API client here.

Thumbnail support for regular pixiv image URLs
Regular pixiv URLs don't have enough information in the URL to derive the
image's URL, so we use the OpenGraph headers for extracting it.
Since we can't block or the UI freezes, we do the request as part of
ImageSizeCheckClient's check.

Pixiv URLs also need a bit of special treatment. Clicking the thumbnail would
open the direct link to the image in the browser, which isn't very useful because
you would get a 403 back (or a small image), so we inject a link to the web
interface for the image.
Contributor

Kovensky commented Aug 3, 2012

Updated to use the og:image meta tag. I think it'd be a good idea to convert this into a class that could check any site that follows the OpenGraph protocol in http://ogp.me/ though.

Owner

psychs commented Aug 8, 2012

d4rkie, is there any progress on this?

d4rkie commented Aug 10, 2012

the staff that can whitelist LimeChat's user-agent is now helping at Comiket... orz
also, agreed with Kovensky to stop using the /dic/320x320/ and use one of the predefined similar-sized images.

Contributor

Kovensky commented Aug 10, 2012

Yeah, I just found out that /dic/320x320/ images are actually generated on the fly... orz

Gonna do changes to always load thumbnails from the mobile versions (which are pre-generated and of sufficient size), should rebase against master and update the commit tomorrow.

Owner

psychs commented Aug 10, 2012

Thanks. I know Comiket is in the highest priority. Let's wait for the staff :)

Yeah, it would be better to get well cached images.

Contributor

Kovensky commented Aug 20, 2012

Haven't done it yet due to laziness / busyness / OSX issues... I should work on this on the tuesday though.

Owner

psychs commented Aug 21, 2012

No problem!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment