Scraper not fetching extra covers

[SkaryMonk]

After updating to 1.0.92 (ComicRack version 0.9.177), the scraper no longer finds any more covers, when I click "(more covers)". I'm simply told there's only a single cover.

A specific example would be Grimm Fairy Tales presents Red Riding Hood One-Shot (http://comicvine.gamespot.com/grimm-fairy-tales-presents-red-riding-hood-one-sho/4000-492312/). Zenescope never published anything in their life which only had one cover, which is what made me suspicious, and ComicVine obviously has more than one cover.

Let me know if you need any further information!

[boshuda]

This is an unfortunate necessity because of Comicvine's decision not to put the covers as fetchable via the API (something about deals with comic publishers). The way CVS used to do it was by scraping the web page. While this had technically been against the rules, Comicvine hadn't been checking for it until they changed their servers. They started blocking the IPs of people caught scraping their web pages. Search their API forum and you'll see a whole drama that unfolded within the last few months. Which is to say this will not be fixed until ComicVine changes their API to pass all of the covers along when requested.

[theotocopulitos]

A work around to this at some point might be to add in the found books list
the direct link to the webpage for each book, so that clicking it would
launch the internet browser to the comic page where the covers are
displayed...

On Fri, Feb 26, 2016 at 1:02 PM, boshuda notifications@github.com wrote:

This is an unfortunate necessity because of Comicvine's decision not to
put the covers as fetchable via the API (something about deals with comic
publishers). The way CVS used to do it was by scraping the web page. While
this had technically been against the rules, Comicvine hadn't been checking
for it until they changed their servers. They started blocking the IPs of
people caught scraping their web pages. Search their API forum and you'll
see a whole drama that unfolded within the last few months.

—
Reply to this email directly or view it on GitHub
#431 (comment)
.

[cbanack]

I'm hoping that they actually add the alternate covers to their official API. They kinda suggested that they'd do it, but it hasn't happened yet.

Once they do, it'll be really easy to get this feature working again.

[giotte]

I know this is incredibly late in coming, but I thought I'd share that ComicVine does now, in fact, provide additional covers via the API. Not sure when it was added, but it definitely is there and works.

All you need to do is update the _query_issue function in cvdb.py so that it looks in the DOM for the "associated_images" element and then append any image URLs to the issue.image_urls_sl list.
You can see an implementation of that here.

This is such a useful feature and I'm just glad that it's working again.

[cbanack]

Thanks for noticing this, giotte! I implemented your changes into a new version of ComicVineScraper (1.0.102) and it seems to be working great. It'll be worth it for people to update to this new version, since it makes "find more covers" feature work properly again, and it will also improve the scraper's ability to "automatically" choose the correct comic book series when scraping (for people who have that feature turned on.)

[unclerandy1969]

I don’t know why, but for some reason I cannot install the latest (final) update. The only thing it seems to write in the scripts folder is a few image files

[cbanack]

Try removing the plugin, restarting comicrack fully (or restart your entire computer), then reinstall the plugin. And then restart comicrack again if needed

I seem to remember that there is a comicrack bug that interfered with installation of plugins sometimes. They was how people worked around it.

[unclerandy1969]

Try removing the plugin, restarting comicrack fully (or restart your entire computer), then reinstall the plugin. And then restart comicrack again if needed

I seem to remember that there is a comicrack bug that interfered with installation of plugins sometimes. They was how people worked around it.

WOW! Thanks, that did the trick