Skip to content

Full stack web application that scrapes news articles from various sources and lets users leave comments on them. Build on a Node.js/Express.js framework, with data stored and retrieved from MongoDB.

Notifications You must be signed in to change notification settings

satchelsmidt/MongoDB_News_Scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

MongoDB_News_Scraper

About

This is a full stack web application that scrapes news articles from The Atlantic Tech Column, displays articles on main app page, and allows users to leave multiple notes on each article. Web scraper used is the Cheerio package. Articles and Notes are saved using MongoDB, and Mongoose is used to make calls between the app and the database.

App is deployed to Heroku

Walkthrough

On application load, the user will be presented with a button prompting them to 'Find new Articles'. Upon pressing this button, the application will use cheerio to scrape all new articles from The Atlantic's tech clumn, and will display each article with the following information:

  • Title
  • Direct link to article
  • Snippet
  • Image

Each article will also include an 'Add Note' button, that pulls up a modal allowing the user to add persistent notes for each specific article.

Here is a gif showing the note-adding function of the app:

Technology Used

  • JavaScript
    • Node.js
    • Express.js
  • Templating Engine
    • Handlebars.js
  • Web Scraper
    • Cheerio
  • HTML
  • CSS
  • MongoDB
    • Mongoose ORM

Future Development

  • Delete already/scraped articles, stop rendering already scraped articles, clear DB on every 'find articles' press
  • Re-push to heroku, not grabbing new articles
  • Load new articles from top, not bottom
  • Change spacing on returned article boxes to avoid weird spacing
  • Change 'find new articles' button to be not ugly
  • Add footer w/ copyright + name + github

About

Full stack web application that scrapes news articles from various sources and lets users leave comments on them. Build on a Node.js/Express.js framework, with data stored and retrieved from MongoDB.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published