Zimit fails to retrieve videos on https://mesquartierschinois.wordpress.com/

[Popolechien]

Scraping a blog with a few youtube/vimeo videos in it: none of them is retrieved. I do get the thumbnail for Youtube video but a generic error message ("an error occured, please try again later") when trying to play. Vimeo does not have thumbnails and simply says "we detected a high number of errors from your connection. To continue please confirm you are a human (and not a spambot)"

[kelson42]

@Popolechien Can you please give a concrete example?

[Popolechien]

Sorry: same blog as in #70. Here are screenshots:

[rgaudin]

Again:

• Task ID/link so we can check logs (exact one) and retrieve ZIM
• Link to online page corresponding to the issue (we can find in-zim one from it)

[Popolechien]

task:
https://farm.youzim.it/pipeline/d1c2f201514f3da67f887df5
Corresponding pages would be
Youtube:
https://mesquartierschinois.wordpress.com/2014/07/17/girona-menamora/
Vimeo:
https://mesquartierschinois.wordpress.com/2014/07/10/will-blog-for-croquetas/

[rgaudin]

There seem to be a problem with video handling on the scraper at the moment. I've opened an upstream ticket at webrecorder/browsertrix-crawler#4

@Popolechien we can't do much until this is resolved but your screenshot might indicate an additional problem on vimeo. We might have hit a quota or something.

[kelson42]

The upstream bug has been fixed, wee need a new webcrawler release.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

@Popolechien Coukd you please verify the bug has been fixed?

[Popolechien]

@kelson42 No idea since we've had another regression to cover it up. See 2661

[rgaudin]

Can you test it with kiwix-serve ? Or share the link to the new Zim so I can check with kiwix-serve ?

[Popolechien]

@rgaudin https://s3.us-west-1.wasabisys.com/org-kiwix-zimit/other/mesquartierschinois.wordpress.com_e4060451.zim

[kelson42]

@Popolechien The zim file is not online anymore and your bug report on Kiwix Android is for the moment not conclusive. Would you be able to make a new one and test with kiwix-serve please?

[Popolechien]

I have no way to run Kiwix-serve locally, so at best I can restart zimit and send you the link?

[Popolechien]

@kelson42 https://s3.us-west-1.wasabisys.com/org-kiwix-zimit/other/mesquartierschinois.wordpress.com_ac060451.zim Here you go

[kelson42]

Thx, still broken as far as I can see.

[rgaudin]

Sorry to reply only after you tested but as noted in mentioned issue, the main issue is openzim/warc2zim#80 which is still open so there were no reason for this to work.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

[kelson42]

@rgaudin Now that openzim/warc2zim#83 is done, what should we do with this ticket?

[rgaudin]

I suggest we close it. Might not be as wanted but behavior has probably changed so we'll reopen once master is released/deployed

[kelson42]

@Popolechien Count on you to retest as soon a new warc2zim release is done.