All data lives in a Postgres database. To run:
scraper/scraper_job.pydownloads all MTA data feeds. Runs every minute (can be more/less frequent; MTA refreshes feed every 30 seconds).
scraper/batch_process.pytakes the downloaded data and determines where trains are. Repeated runs with
--job-namewill pick up from where the last run left off. Runs every minute. Can be used to backfill data as well.
scraper/importer.py data/nycto store static GTFS data into the database
scraper/import_shapes.py data/nyc/shapes.txt data/nyc/stops.txtgenerates the subway map given all routes and stations
uvicorn web.app:appto start the web server
To run the code, environment variables need to be set.
NYC_MTA_API_KEYis only required for scraper.py