Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bring back Facebook by scraping #886

snarfed opened this issue Sep 10, 2019 · 0 comments


Copy link

@snarfed snarfed commented Sep 10, 2019 is a little-known "lite" version of Facebook's full webapp with no JS and fairly simple HTML. it requires login, specifically c_user and xs cookies, but it's eminently scrapeable. already scrapes it to generate Atom feeds. apart from how distasteful it is to scrape with login cookies, we could scrape it like Instagram to bring back Facebook backfeed!

...sadly, FB's blocking is better than IG's. i actually implemented the scraping and extracted posts, comments, and likes/reactions, but i haven't been able to fetch users' timelines consistently. after one or two requests, FB consistently starts redirecting requests to /login.php, even with all cookies that gives me, fully spoofed User-Agent, and fetching from the same IP I logged in from. maybe browser fingerprinting? got me. this is where i stop digging. scraping, ugh.


@snarfed snarfed closed this Sep 10, 2019
snarfed added a commit that referenced this issue Sep 11, 2019

for #886. IN PROGRESS.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
1 participant
You can’t perform that action at this time.