Skip to content

Exploring the world of possibilities afforded by the intersection of data engineering and the Internet of Things (IoT).

Notifications You must be signed in to change notification settings

chiralevy/Real-time-ETL-Pipeline-for-IoT-System

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Real-time ETL Pipeline for IoT System

General Objectives:

  • Using the onion omega 2 (IOT device), take pictures every 15-30 seconds, obtain GPS coordinates, and record audio
  • Capture multimodal data to onion's SD card, store until the internet is available (then send off)
  • Save (via FTP) to the SFTP server
  • Pick up data (from the server)
  • Process Files to the Database
  • Develop webpage to display IOT data and insights

Project Timeline (by week):

  • June 3-7: Orientation and Setup
  • June 10-14: Preliminary Design/Formats and Capture to SD card
  • June 17-21: Internet Availability Check and send to SFTP
  • June 24-28: Coding File pickup
  • July 1-5: Database Intro and Schema Design
  • July 8-12: ETL to Database
  • July 15-19: Job Scheduling and UI Review/Define/Research
  • July 22-26: Grace Period
  • If time: Facial Recognition; Speech to Text/Geoplotting/Pinpoints/Roles; Data Analysis

Feel free to reach out if you would like to discuss this project in depth.

About

Exploring the world of possibilities afforded by the intersection of data engineering and the Internet of Things (IoT).

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published