Skip to content
This repository has been archived by the owner on Jun 27, 2024. It is now read-only.

Does this script still scrape in a disallowed manner? #33

Closed
smesseim opened this issue Aug 14, 2018 · 1 comment
Closed

Does this script still scrape in a disallowed manner? #33

smesseim opened this issue Aug 14, 2018 · 1 comment

Comments

@smesseim
Copy link

The README states that this script breaks the Golden Rules. Is this still correct? If so, which of the interactions in this script are disallowed? Perhaps there could be a workaround (e.g. instead of scraping better.php, the user could download the better.php page and pass that as an input to this script). I can work on that if I know where the issue lies.

@Mechazawa
Copy link
Owner

Mechazawa commented Aug 22, 2018

nope it hasn't for quite a while https://github.com/Mechazawa/REDBetter-crawler/search?q=request_html&unscoped_q=request_html

it used to though. Thanks for the heads up!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants