Skip to content

Smartproxy/reddit-python-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 

Repository files navigation

Reddit Scraper

Scrape Reddit utilising Smartproxy's Web Scraping API

Dependencies

BeautifulSoup

Authentication

Once you have an active Web Scraping API subscription, you can try sending a request right from the dashboard Web Scraping API > API playground method tab simply by clicking on Send Request. You will also see an example of curl request generated on the right.

This Pyhton code example uses Base64 encoded user:pass authentication.

Parser type Example location Download
HTML to JSON reddit_python_scraper.py curl https://raw.githubusercontent.com/Smartproxy/reddit-python-scraper/blob/main/reddit_python_scraper.py > reddit_python_scraper.py

HTML to JSON

This Python script extracts Subreddit details, post data and comments straight from the HTML of Reddit post page and saves them to a JSON file.

About

Scrape Reddit utilising Smartproxy's Web Scraping API

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages