Skip to content

This is an webscrapper that fetches news from various news websites. A git workflow triggers the scrapper that updates the database.

Notifications You must be signed in to change notification settings

juliocesardemoraes/PythonScraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

52 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

🕷️PythonScraper🕷️

Description

Backend for scraping news websites, with sentiment analysis.

Tech Stack

  • Python
  • Mongo
  • Scrapy
  • Nltk(sentiment analysis)
  • TDD(pytest)
  • Linting(pylint)

How to Run IT

Running Crawler with scrapy

scrapy crawl article

Testing With pytest

python -m pytest tests/

Extra

-- In Progress

About

This is an webscrapper that fetches news from various news websites. A git workflow triggers the scrapper that updates the database.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published