Skip to content

Crawler script to collect all the news from every website

License

Notifications You must be signed in to change notification settings

superXdev/opennews-crawler

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenNews Crawler (scraping)

A small program to automated update news for every single time, this script was coded in python and using bs4 module for scraping task. News source from:

Installation

  1. Clone this repo
  2. Install all requirements module
pip install -r requirements.txt
  1. Set your database configuration (db.py)
  2. Make sure the database was migrated from API project
  3. Set schedule time (main.py)
  4. Run
python main.py

That's it!

Contact me

About

Crawler script to collect all the news from every website

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages