31st project of the Trybe Web Development course.
Tech News is a Python project that uses Beautiful Soup to perform web scraping on Trybe's blog and fetch tech news and articles, storing them in a MongoDB database.
ℹ️ I wrote tests for the functions implemented by Trybe. These tests can be found in the
testssubdirectory.
The project was developed in Python and tested with Pytest. The database used was MongoDB and the connection and interaction to it was established through the PyMongo library. Flake8 ensured code quality while Beautiful Soup performed web scraping.
- Collect news and articles from Trybe's blog using web scraping and store them in a MongoDB database.
- Search news by title.
- Search news by date.
- Browse news by category.
- List the top 5 categories with the most news.
To run the project locally, follow the steps below.
- Clone the repository.
git clone git@github.com:garciaagui/tech-news.git
- Navigate to the root of the project.
cd tech-news/
- Create the virtual environment.
python3 -m venv .venv
- Activate the virtual environment.
source .venv/bin/activate
- Note that at the beginning of the terminal line there will be
(.venv), as in the example below.
(.venv) gui@gui-desktop:~/Trybe/tech-news$
- To deactivate the virtual environment, run the command
deactivate. Remember to activate it again when you return to the project.
- Install dependencies in the virtual environment.
python3 -m pip install -r dev-requirements.txt
- If you don't have MongoDB installed locally, launch it via Docker.
docker-compose up -d mongodb
- ℹ️ Database name is
tech_news.
- Run the following command to access the menu.
tech-news-analyzer
ℹ️ For additional instructions, click here.
- 🧪 To run all tests, execute the command below.
python3 -m pytest
- 🧪 To run only one test file, follow the example below.
python3 -m pytest tests/reading_plan/test_reading_plan.py
- 🧪 To run only one specific test, follow the example below.
python3 -m pytest -k test_reading_plan_group_news_with_valid_input
- If you wish to manually test directly in the modules where the functions were implemented, follow the example below.
python3 -m tech_news.scraper.py
"Trybe is a future school for anyone who wants to improve their lives and build a successful career in technology, where the person only pays when they get a good job."
"The program features over 1,500 hours of online classes covering introduction to software development, front-end, back-end, computer science, software engineering, agile methodologies, and behavioral skills."
Project developed by Guilherme Garcia. Below are my social networks and means of contact. 🤘
