The Automated News Web Scraper is an automated Python script designed to run daily, collecting and aggregating news articles from tech website. This project provides a convenient way to stay updated with the latest news by scraping relevant information from specified website.
- Automated Daily Execution with Selenium: Schedule the script to run daily and fetch the latest news.
- Website Aggregation: Scrape news articles from specified websites using XPath.
- Data Storage in CSV file: Store scraped data in a structured format for analysis or future reference.
- Dependencies: All necessary dependencies are listed in the
requirements.txt
file for easy installation.
- Python 3.x
- Pip (Python Package Installer)
-
Clone the repository:
git clone https://github.com/babicmila/news-web-scraper.git cd news-web-scraper
-
Install dependencies:
pip install -r requirements.txt
-
Run the script:
python news-headlines.py
-
Schedule Daily Execution (Optional):
To run the script daily, set up a scheduled task (Windows) or crontab job (macOS).
For Windows users:
- Search for “Task Scheduler”.
- Go to Actions > Create Task, and give a name to the task
- Go to Actions > New
- In Program/Script, add the path to the Python executable file.
- In the "Add arguments (optional)” box, add the name of your Python file (news-headlines.exe).
- In the "Start in (optional)" box, add the location of that Python file.
- Go to “Triggers” > New, and schedule the script to run daily and choose the time you prefer