Data Flow

Steps

Step 1

The data acquisitor [insert folder link] is responsible for

• Scraping the outside graph from OSM
• Scraping building outlines from ESIM
• Extracting polygons from the floorplans in the FMS Website
  • Link each polygon with a room name, type, etc.
    • Full type definition
  • Generate graphs from the polygons
  • Useful file: https://github.com/ScottyLabs/cmumaps-data-visualization-deprecated/blob/main/public/python/api/walkway_detection.py
• And store all data in JSON files in S3 buckets

Step 2

The deserializer will read the JSON data from S3 bucket and populate the database in Staging.

Step 3

In this step, the visualizer in Staging allows us to visually verify the integrity of the graph and polygons. Then, the visualizer allows the floor to be placed in geo coordinates.

[TODO: more explanation coming soon]

Note: we want to do this step in staging so users will not see unplaced floor data.

Step 4

The serializer will read from the database and serialize the data to JSONs in S3 bucket.

Step 5

The deserializer will read the JSON data from S3 bucket and populate the database in Prod.

Step 6

The visualizer in Prod allows us to quickly add POIs (points of interest) that users can immediately see.

Excalidraw Files

Newest

cmumaps-data-flow.excalidraw.json