Skip to content

Created an app with Flask to display data retrieved by web scraping with Pandas, Beautiful Soup, and Splinter using Jupyter Notebook and stored in mongoDB. End user can easily retrieve the latest information with a simple click.

Notifications You must be signed in to change notification settings

LeeProut/web-scraping-challenge

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

37 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Mission to Mars

  • web scraping with Pandas, Beautiful Soup, and Splinter using Jupyter Notebook
  • creating an app to display the data retrieved using MongoDB with Flask
  • end user can easily retrieve the latest information with a simple click

Web Scraping Demo

View web scraping in action here.

YouTube video of web scraping


Source sites:


Web scraping

Initial web scraping was done in a Jupyter notebook that can be viewed here.

MongoDB and Flask Application

The Jupyter notebook was converted into a Python script that can be viewed here. It defined a scrape function to execute scraping code and return a Python dictionary that contains the scraped data.

Created an app using Flask, with code that can be viewed here. The root route queries the Mongo database and passes the Mars data into an HTML template to display on a webpage. The /scrape route imports the Python script and calls the scrape function to retrieve new data when the user clicks the "Scrape New Data" button on the page.

The finished product is interactive, with a clean design:

Mission to Mars

Mars Hemispheres Images

The information updates with each click of the "Scrape New Data" button. The most striking retrieval is the "Featured Mars Image."

Featured Image retrieved updated Featured Image
Mission to Mars Mission to Mars

The design is responsive to screen size:

Responsive Design Display

About

Created an app with Flask to display data retrieved by web scraping with Pandas, Beautiful Soup, and Splinter using Jupyter Notebook and stored in mongoDB. End user can easily retrieve the latest information with a simple click.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published