Skip to content

apurvsinghgautam/ForumScrapy

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 

ForumScrapy

This is a Web Scraping application built on Flask. This web application scrapes forum posts from the Bitshacking hack forum. It uses Elasticsearch as the database to store the forum data and Kibana is used for visualizing data from Elasticsearch. Kibana is also used for visualizing data from the Elasticsearch database.

ForumScrapy Homepage

Prerequisites

  • Python
  • Flask
  • Requests
  • Elasticsearch 5.6
  • BeautifulSoup
  • Kibana

Usage

  • Enter any Socks proxy in the code
  • Enter your Bitshacking forum username and password (md5) in the code
  • Refresh Cookies if needed
  • Enter the Flask secret key in the code
  • Run the ForumScrapy.py file
  • Open 127.0.0.1:5000 in any browser
  • Open 127.0.0.1:5601 in any browser to see the Kibana dashboard

Screenshots

ForumScrapy Forums

ForumScrapy Detailed

ForumScrapy Members

Elasticsearch forum data

Kibana Dashboard

Contributors

About

Flask application for scraping forum data from Bitshacking forum

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published