Skip to content
Permalink
Branch: master
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
648 lines (481 sloc) 27.9 KB
# 3-03-2019:
# Refactoring the code by only using simple dictionary iterations to get this project done:
# Helpful Links:
# https://stackoverflow.com/questions/2733813/iterating-through-a-json-object
# https://stackoverflow.com/questions/34951448/issues-iterating-through-json-list-in-python/34951507
# I was able to pretty much return everything I want in exact lists.
# I could now pump these lists into a panda data frame, and then convert that into an Excel .xlsx file and SQL database.
# Once I'm able to convert into Excel and SQL, then I will remove the current print statements present, and post updates to Twitter every
# 5 mins from 7 to 9 AM EST, and from 4 to 6 PM EST
# 2-25-2019:
# After examining the statuses, I feel like its very necessary to make sure that each train has a current status field, since it IS present within the
# block data currently present.
# This might take a re-thinking in terms of maybe just dealing with a panda that parses the json data instead since I don't think the
# current data is actually accurate, and would rather work with a panda instead
# Better yet, it might be better to just use JSON itself from Python 3's docs to prevent any loss of data:
# https://docs.python.org/3/library/json.html
# 2-19-2019:
# After looping through the available 'Current' train data, I realized it's only giving me one train's worth of data,
# which is usually just the express train.
# This is either an error on my part which the portion of the data involving it.
# The schedule train data appears to be sufficient enough for this project to be quite honest.
# This is because its the same kind of info that you would normally care about, and to track each of the trains along the route anyway.
# I might just focus on this section of the data for this project anyway, since I'd like to monitor how the trains are moving along a given
# route in the course of 4 hours of the the day (7:00 AM to 9:00 AM, and 4:00 PM to 6:00 PM)
# 2-11-2019:
# Need to fix the list index out of range error, but mostly transferred my code from Repl.it to this project
# I was able to also move everything into individual functions, and write the data to a related text file as well for reference
# I would like to write to .xlsx before scaling it to a SQL database
# 1-28-2019:
# I was able to successfully move all data into related lists, and print each data point
# I will move this into a .csv or .xlsx file next week, and then scale it to be inside of a SQL database that could be
# later queried depending on the day
# Once I place this into an Excel file, I'll be able to understand how I want to better map the data into individual tables and rows.
# Definitely looking forward to the outcome of this so far.
# 1-21-2019:
# I made even more progress in understanding how to further manipulate block.get() to rip out sections of the data
# and to assign them to variables.
# At this point, I have to somehow rip out the keys and values of each of the array elements in the '' list
# After that, I can then list the train's current stop + each scheduled stop in separate rows
# Once that is complete, I will then add this to a .xlsx file, and then later a SQL database.
# Related goals from an IRC member from #SQL Freenode channel on how to approach this:
# "If you'd want a table of routes, then a table of stops, then a table of what stops are on what routes.
# "then a table linking that one to stop times for each route at the stop"
# So basically, it's a three tables: 1. Table of routes, 2. Table of stops, 3. Table of stops on the routes, 4. Table that links table of stops to
# table of stops on the routes
# 1-15-2019:
# I was able to get a little bit closer with manipulating the available data by looking at the output in Featherpad (Linux)
# and realized I could just rip out data in variables similar to the Jupyter Notebook project example
# The only issue I came across was the fact that when you start indexing into it with the block[][] statements below,
# it returns a LIST of a dictionary of keys and values, and some of the values are actually keys themselves.
# Talk about confusing! With that in mind, I adapted the following Stackoverflow solution to changing it into a dictionary (second answer):
# https://stackoverflow.com/questions/5236296/how-to-convert-list-of-dict-to-dict
# I still feel like I might be overthinking this, but I'm getting closer. Also, it also hit me that you can't judge the available data by
# being late or present by the fact that it contains a tripUpdate or vehicle fields, because EVERY train has both. They are just present
# in separate id's within the data so a train will contain both a tripUpdate AND a vehicle section.
# I guess you can only judge if its on time based on the milliseconds provided in the data. That would probably even be more recognizable
# once I'm able to change this into an Excel .xlsx file for further analysis to later then make into a Sqllite database as well.
# 1-8-2019:
# I was finally able to work on this project again since I've been sick for the last two weeks.
# Since my implementation is much harder than the "Example Links To Model" second link, I'm relying on
# print statements to get me closer to imbedded key values. This approach is working though.
# I was even able to further clean up the code by only using .get('vehicle', {}) and .get('tripUpdate, {}) to rip the current data
# Based on what I researched, and the values being returned, the following two scenarios should be true:
# If vehicle data is returned, then the train is on time at a stop, or is on time in transit to another stop.
# If tripUpdate data is returned, then there has been a delay in the system.
# Both data sets are important because it would be a good idea to keep good tabs with how many vehicles are on time vs not on time
# which can be simplified into variables called vehiclesOnTime and vehiclesLate
# 12-12-2018:
# I tried using pprint to see what's going on with this data
# The current issue is that if the 'vehicle' field isn't present, then the printing code is never run.
# I attempted adding the if statement to make sure its present before printing
# Example Links To Model:
# https://nbviewer.jupyter.org/url/nikhilvj.co.in/files/gtfsrt/write1.ipynb
# https://nbviewer.jupyter.org/url/nikhilvj.co.in/files/gtfsrt/locations.ipynb
import glob
from google.protobuf import json_format
from google.transit import gtfs_realtime_pb2
from google.protobuf.json_format import MessageToDict
import requests
import datetime
import pprint
import pandas as pd
import time
import xlsxwriter
def getRequest(MTADataFeed):
r = requests.get(MTADataFeed)
return r
def createFeed(r):
feed = gtfs_realtime_pb2.FeedMessage()
return feed
def convertToDictFeed(r, feed):
feed.ParseFromString(r.content)
dictFeed = MessageToDict(feed)
return dictFeed
def pprintDictFeed(dictFeed):
pprint.pprint(dictFeed)
def createTrainDataLists(dictFeed):
print('\n\nLooping through dictFeed list: ')
i = 0
trainScheduleDataList = []
trainCurrentStopDataList = []
# Looping Through List:
for item in dictFeed['entity']:
# print(str(item))
if i % 2 == 0:
trainScheduleDataList.append(item)
else:
trainCurrentStopDataList.append(item)
i += 1
return trainScheduleDataList, trainCurrentStopDataList
def printTrainDataLists(trainScheduleDataList, trainCurrentStopDataList):
print('\n\nChecking trainScheduleDataList: ')
for item in trainScheduleDataList:
print('\n\n')
print(item)
print('\ntype(item) = ' + str(type(item)))
print('\n\nChecking trainCurrentStopDataList: ')
for item in trainCurrentStopDataList:
print('\n\n')
print(item)
print('\ntype(item) = ' + str(type(item)))
def getIDsAndTripUpdate(trainScheduleDataList):
# print('\n\nPrinting trainScheduleDataList: ')
# print(str(trainScheduleDataList))
# print('\n\ntype(trainScheduleDataList = ' + str(type(trainScheduleDataList)))
trainScheduleIDList = []
trainScheduleTripUpdateList = []
print('\n\nCycling Through trainScheduleDataList: ')
for dictItem in trainScheduleDataList:
# print('\n\n')
# print(dictItem)
# print('type(dictItem) = ' + str(type(dictItem)))
# print("dictItem['id'] = " + str(dictItem['id']))
trainScheduleIDList.append(dictItem['id'])
# print("dictItem['tripUpdate'] = " + str(dictItem['tripUpdate']))
trainScheduleTripUpdateList.append(dictItem['tripUpdate'])
# print('Displaying Keys And Values in dictItems.items(): ')
# for key, value in dictItem.items():
# print('\n\n{}: {}'. format(key, value))
return trainScheduleIDList, trainScheduleTripUpdateList
def getTrainScheduleTripListAndStopTimeUpdateList(trainScheduleTripUpdateList):
trainScheduleTripList = []
trainScheduleStopTimeUpdateList = []
print('\n\nNow looping through trainscheduleTripupdateList:')
for dictItem in trainScheduleTripUpdateList:
trainScheduleTripList.append(dictItem['trip'])
trainScheduleStopTimeUpdateList.append(dictItem['stopTimeUpdate'])
print('\n\nChecking trainScheduleStopTimeUpdateList: ')
for item in trainScheduleStopTimeUpdateList:
print(item)
print('type(item) = ' + str(type(item)))
return trainScheduleTripList, trainScheduleStopTimeUpdateList
def obtainTrainScheduleTripListData(trainScheduleTripList):
# print('\n\nNow checking trainScheduleTripList:')
trainScheduleTripIDList = []
trainScheduleTripStartDateList = []
trainScheduleTrainRouteIDList = []
for dictItem in trainScheduleTripList:
trainScheduleTripIDList.append(dictItem['tripId'])
euroStyleDate = str(dictItem['startDate'])
year = euroStyleDate[0:4]
month = euroStyleDate[4:6]
day = euroStyleDate[6:8]
americanStyleDate = month + '-' + day + '-' + year
trainScheduleTripStartDateList.append(americanStyleDate)
trainScheduleTrainRouteIDList.append(dictItem['routeId'])
return trainScheduleTripIDList, trainScheduleTripStartDateList, trainScheduleTrainRouteIDList
def obtainTrainScheduleStopTimeUpdateListData(trainScheduleStopTimeUpdateList):
trainScheduleArrivalList = []
trainScheduleDepartureList = []
trainScheduleStopIDList = []
trainScheduleRelationshipList = []
print('\n\nNow checking trainScheduleStopTimeUpdateList: ')
for list in trainScheduleStopTimeUpdateList:
# print(listItem)
# print('type(listItem) = ' + str(type(listItem)))
for dictItem in list:
print(dictItem)
trainScheduleArrivalList.append(dictItem['arrival'])
trainScheduleDepartureList.append(dictItem['departure'])
trainScheduleStopIDList.append(dictItem['stopId'])
trainScheduleRelationshipList.append(dictItem['scheduleRelationship'])
# Loop Through trainScheduleArrivalList To Obtain Arrival Time List:
trainScheduleArrivalTimeList = []
trainScheduleDepartureTimeList = []
print('Going Through trainScheduleArrivalList: ' )
for dictItem in trainScheduleArrivalList:
print(dictItem['time'])
print("type(dictItem['time']) = " + str(type(dictItem['time'])))
trainScheduleArrivalTimeList.append(dictItem['time'])
print('Going Through trainScheduleDepartureList: ' )
for dictItem in trainScheduleDepartureList:
print(dictItem['time'])
print("type(dictItem['time']) = " + str(type(dictItem['time'])))
trainScheduleDepartureTimeList.append(dictItem['time'])
# Checking Lists:
print('\n\nChecking trainScheduleArrivalTimeList: ')
for item in trainScheduleArrivalTimeList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainScheduleDepartureTimeList: ')
for item in trainScheduleDepartureTimeList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainScheduleStopIDList: ')
for item in trainScheduleStopIDList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainScheduleRelationshipList: ')
for item in trainScheduleRelationshipList:
print(item)
print('type(item) = ' + str(type(item)))
return trainScheduleArrivalTimeList, trainScheduleDepartureTimeList, trainScheduleStopIDList, trainScheduleRelationshipList
def processTrainCurrentStopDataList(trainCurrentStopDataList):
trainCurrentStopIDList = []
trainCurrentStopVehicleDataList = []
trainCurrentStopVehicleTripDataList = []
trainCurrentStopTripIDList = []
trainCurrentStopStartDateList = []
trainCurrentStopRouteIDList = []
trainCurrentStopCurrentStopSequenceList = []
trainCurrentStopCurrentStatusList = []
trainCurrentStopTimestampList = []
print('\n\nChecking trainCurrentStopDataList):')
print(str(trainCurrentStopDataList))
print('type(trainCurrentStopDataList) = ' + str(type(trainCurrentStopDataList)))
print('\n\nCycling through trainCurrentStopDataList: ')
for dictItem in trainCurrentStopDataList:
print(dictItem)
print('type(dictItem) = ' + str(type(dictItem)))
print("dictItem['id'] = " + str(dictItem['id']))
trainCurrentStopIDList.append(dictItem['id'])
print("dictItem['vehicle'] = " + str(dictItem['vehicle']))
trainCurrentStopVehicleDataList.append(dictItem['vehicle'])
trainCurrentStopVehicleTripDataList = []
trainCurrentStopTripIDList = []
trainCurrentStopStartDateList = []
trainCurrentStopRouteIDList = []
trainCurrentStopCurrentStopSequenceList = []
trainCurrentStopCurrentStatusList = []
trainCurrentStopTimestampList = []
print('\n\nChecking trainCurrentStopVehicleDataList: ')
for dictItem in trainCurrentStopVehicleDataList:
print(dictItem)
print('type(dictItem) = ' + str(type(dictItem)))
trainCurrentStopVehicleTripDataList.append(dictItem['trip'])
trainCurrentStopCurrentStopSequenceList.append(dictItem['currentStopSequence'])
trainCurrentStopCurrentStatusList.append(dictItem['currentStatus'])
trainCurrentStopTimestampList.append(dictItem['timestamp'])
print('\n\nChecking trainCurrentStopVehicleTripDataList: ')
for dictItem in trainCurrentStopVehicleTripDataList:
print(dictItem)
print('type(dictItem) = ' + str(type(dictItem)))
trainCurrentStopTripIDList.append(dictItem['tripId'])
euroStyleDate = str(dictItem['startDate'])
year = euroStyleDate[0:4]
month = euroStyleDate[4:6]
day = euroStyleDate[6:8]
americanStyleDate = month + '-' + day + '-' + year
trainCurrentStopStartDateList.append(americanStyleDate)
trainCurrentStopRouteIDList.append(dictItem['routeId'])
print('\n\nChecking trainCurrentStopIDList: ')
for item in trainCurrentStopIDList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainCurrentStopCurrentStopSequenceList: ')
for item in trainCurrentStopCurrentStopSequenceList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainCurrentStopCurrentStatusList: ')
for item in trainCurrentStopCurrentStatusList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainCurrentStopTimestampList: ')
for item in trainCurrentStopTimestampList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainCurrentStopTripIDList: ')
for item in trainCurrentStopTripIDList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainCurrentStopStartDateList: ')
for item in trainCurrentStopStartDateList:
print(item)
print('type(item) = ' + str(type(item)))
print('\n\nChecking trainCurrentStopRouteIDList: ')
for item in trainCurrentStopRouteIDList:
print(item)
print('type(item) = ' + str(type(item)))
return trainCurrentStopIDList, trainCurrentStopTripIDList, trainCurrentStopStartDateList, trainCurrentStopRouteIDList, trainCurrentStopCurrentStopSequenceList, trainCurrentStopCurrentStatusList, trainCurrentStopTimestampList
def countVehicles(feed):
# Count vehicles present in feed
content = ''
count = 0
# Loop through feed.entity:
for entity in feed.entity:
if entity.HasField('vehicle'):
count += 1
return count
def checkLength(listItem):
return len(listItem)
def convertToPanda1(trainScheduleArrivalTimeList, trainScheduleDepartureTimeList, trainScheduleStopIDList, trainScheduleRelationshipList):
masterList = [trainScheduleArrivalTimeList, trainScheduleDepartureTimeList, trainScheduleStopIDList, trainScheduleRelationshipList]
print('\n\nInside convertToPanda1(): ')
print('\n\nChecking masterList: ')
for listItem in masterList:
arrayLength = checkLength(listItem)
print('arrayLength = ' + str(arrayLength))
d = {'trainScheduleArrivalTime': trainScheduleArrivalTimeList,
'trainScheduleDepartureTime': trainScheduleDepartureTimeList,
'trainScheduleStopID': trainScheduleStopIDList,
'trainScheduleRelationship': trainScheduleRelationshipList}
schedule_df = pd.DataFrame(data=d)
print('\n\nPrinting schedule_df: ')
print(schedule_df)
return schedule_df
def convertToPanda2(trainScheduleIDList, trainScheduleTripIDList, trainScheduleTripStartDateList, trainScheduleTrainRouteIDList, trainCurrentStopIDList, trainCurrentStopTripIDList, trainCurrentStopStartDateList, trainCurrentStopRouteIDList, trainCurrentStopCurrentStopSequenceList, trainCurrentStopCurrentStatusList, trainCurrentStopTimestampList):
masterList1 = [trainScheduleIDList, trainScheduleTripIDList, trainScheduleTripStartDateList, trainScheduleTrainRouteIDList]
print('\n\nInside convertToPanda2(): ')
print('\n\nChecking masterList1: ')
for listItem in masterList1:
arrayLength = checkLength(listItem)
print('arrayLength = ' + str(arrayLength))
masterList2 = [trainCurrentStopIDList, trainCurrentStopTripIDList, trainCurrentStopStartDateList, trainCurrentStopRouteIDList, trainCurrentStopCurrentStopSequenceList, trainCurrentStopCurrentStatusList, trainCurrentStopTimestampList]
print('\n\nInside convertToPanda2(): ')
print('\n\nChecking masterList2: ')
for listItem in masterList2:
arrayLength = checkLength(listItem)
print('arrayLength = ' + str(arrayLength))
d1 = {'trainScheduleID': trainScheduleIDList,
'trainScheduleTripID': trainScheduleTripIDList,
'trainScheduleTripStartDate': trainScheduleTripStartDateList,
'trainScheduleTrainRouteID': trainScheduleTrainRouteIDList}
currentstop_df1 = pd.DataFrame(data=d1)
print('\n\nPrinting currentstop_df1: ')
print(currentstop_df1)
d2 = {'trainCurrentStopID': trainCurrentStopIDList,
'trainCurrentStopTripID': trainCurrentStopTripIDList,
'trainCurrentStopStartDate': trainCurrentStopStartDateList,
'trainCurrentStopRouteID': trainCurrentStopRouteIDList,
'trainCurrentStopCurrentStopSequence': trainCurrentStopCurrentStopSequenceList,
'trainCurrentStopCurrentStatus': trainCurrentStopCurrentStatusList,
'trainCurrentStopTimestamp': trainCurrentStopTimestampList}
currentstop_df2 = pd.DataFrame(data=d2)
print('\n\nPrinting currentstop_df2: ')
print(currentstop_df2)
return currentstop_df1, currentstop_df2
def writeExcelRows(panda, worksheet):
# Start on row 1:
# x = 1
# for index, row in panda.iterrows():
# worksheet.write(x, panda.index, row['panda'], centerAlign)
print('\n\nChecking panda.index: ')
print(panda.index)
def dumpToXLSX(schedule_df, currentstop_df1, currentstop_df2):
currentDate = datetime.datetime.now().strftime('%m-%d-%y, %I-%M-%S %p')
xlsxWorkbookName = 'MTA Data (' + str(currentDate) + ').xlsx'
workbook = xlsxwriter.Workbook(xlsxWorkbookName)
scheduledWorksheet = workbook.add_worksheet()
currentStopWorksheet1 = workbook.add_worksheet()
currentStopWorksheet2 = workbook.add_worksheet()
centerAlign = workbook.add_format({'align': "center"})
boldCenterAlign = workbook.add_format({'align': "center", 'bold': True})
# schedule_df column headers
# trainScheduleArrivalTimeList, trainScheduleDepartureTimeList, trainScheduleStopIDList, trainScheduleRelationshipList]
scheduledWorksheet.write(0, 0, 'Arrival Date', boldCenterAlign)
scheduledWorksheet.write(0, 1, 'Arrival Time', boldCenterAlign)
scheduledWorksheet.write(0, 2, 'Departure Date', boldCenterAlign)
scheduledWorksheet.write(0, 3, 'Departure Time', boldCenterAlign)
scheduledWorksheet.write(0, 4, 'Stop ID', boldCenterAlign)
scheduledWorksheet.write(0, 5, 'Schedule Relationship', boldCenterAlign)
# Trying to write this out to a function for all available pandas
writeExcelRows(schedule_df, scheduledWorksheet)
# Row variable, x:
x = 1
# Write each row from schedule_df panda:
for index, row in schedule_df.iterrows():
posix_arrival_time = int(row['trainScheduleArrivalTime'])
est_arrival_date = time.strftime('%m-%d-%Y', time.localtime(posix_arrival_time))
est_arrival_time = time.strftime('%I:%M:%S %p', time.localtime(posix_arrival_time))
scheduledWorksheet.write(x, 0, est_arrival_date, centerAlign)
scheduledWorksheet.write(x, 1, est_arrival_time, centerAlign)
posix_departure_time = int(row['trainScheduleDepartureTime'])
est_departure_date = time.strftime('%m-%d-%Y', time.localtime(posix_departure_time))
est_departure_time = time.strftime('%I:%M:%S %p', time.localtime(posix_departure_time))
scheduledWorksheet.write(x, 2, est_departure_date, centerAlign)
scheduledWorksheet.write(x, 3, est_departure_time, centerAlign)
scheduledWorksheet.write(x, 4, row['trainScheduleStopID'], centerAlign)
scheduledWorksheet.write(x, 5, row['trainScheduleRelationship'], centerAlign)
x += 1
# currentstop_df1:
# trainScheduleIDList, trainScheduleTripIDList, trainScheduleTripStartDateList, trainScheduleTrainRouteIDList
currentStopWorksheet1.write(0, 0, 'ID', boldCenterAlign)
currentStopWorksheet1.write(0, 1, 'Trip ID', boldCenterAlign)
currentStopWorksheet1.write(0, 2, 'Start Date', boldCenterAlign)
currentStopWorksheet1.write(0, 3, 'Route ID', boldCenterAlign)
# Reset row variable to 1
x = 1
# Write each row from currentstop_df1 panda:
for index, row in currentstop_df1.iterrows():
currentStopWorksheet1.write(x, 0, row['trainScheduleID'], centerAlign)
currentStopWorksheet1.write(x, 1, row['trainScheduleTripID'], centerAlign)
currentStopWorksheet1.write(x, 2, row['trainScheduleTripStartDate'], centerAlign)
currentStopWorksheet1.write(x, 3, row['trainScheduleTrainRouteID'], centerAlign)
x += 1
# currentstop_df2:
# trainCurrentStopIDList, trainCurrentStopTripIDList, trainCurrentStopStartDateList, trainCurrentStopRouteIDList, trainCurrentStopCurrentStopSequenceList, trainCurrentStopCurrentStatusList, trainCurrentStopTimestampList
currentStopWorksheet2.write(0, 0, 'ID', boldCenterAlign)
currentStopWorksheet2.write(0, 1, 'Trip ID', boldCenterAlign)
currentStopWorksheet2.write(0, 2, 'Start Date', boldCenterAlign)
currentStopWorksheet2.write(0, 3, 'Route ID', boldCenterAlign)
currentStopWorksheet2.write(0, 4, 'Current Stop Sequence', boldCenterAlign)
currentStopWorksheet2.write(0, 5, 'Current Status', boldCenterAlign)
currentStopWorksheet2.write(0, 6, 'Current Date', boldCenterAlign)
currentStopWorksheet2.write(0, 7, 'Timestamp', boldCenterAlign)
# Reset row variable to 1
x = 1
# Write each row from currentstop_df1 panda:
for index, row in currentstop_df2.iterrows():
currentStopWorksheet2.write(x, 0, row['trainCurrentStopID'], centerAlign)
currentStopWorksheet2.write(x, 1, row['trainCurrentStopTripID'], centerAlign)
currentStopWorksheet2.write(x, 2, row['trainCurrentStopStartDate'], centerAlign)
currentStopWorksheet2.write(x, 3, row['trainCurrentStopRouteID'], centerAlign)
currentStopWorksheet2.write(x, 4, row['trainCurrentStopCurrentStopSequence'], centerAlign)
currentStopWorksheet2.write(x, 5, row['trainCurrentStopCurrentStatus'], centerAlign)
posix_timestamp = int(row['trainCurrentStopTimestamp'])
est_current_date = time.strftime('%m-%d-%Y', time.localtime(posix_timestamp))
est_timestamp = time.strftime('%I:%M:%S %p', time.localtime(posix_timestamp))
currentStopWorksheet2.write(x, 6, est_current_date, centerAlign)
currentStopWorksheet2.write(x, 7, est_timestamp, centerAlign)
x += 1
# Set Column Widths:
scheduledWorksheet.set_column(0, 0, 40)
scheduledWorksheet.set_column(1, 1, 40)
scheduledWorksheet.set_column(2, 2, 40)
scheduledWorksheet.set_column(3, 3, 40)
currentStopWorksheet1.set_column(0, 0, 40)
currentStopWorksheet1.set_column(1, 1, 40)
currentStopWorksheet1.set_column(2, 2, 40)
currentStopWorksheet1.set_column(3, 3, 40)
currentStopWorksheet2.set_column(0, 0, 40)
currentStopWorksheet2.set_column(1, 1, 40)
currentStopWorksheet2.set_column(2, 2, 40)
currentStopWorksheet2.set_column(3, 3, 40)
currentStopWorksheet2.set_column(4, 4, 40)
currentStopWorksheet2.set_column(5, 5, 40)
currentStopWorksheet2.set_column(6, 6, 40)
workbook.close()
return xlsxWorkbookName
def main():
MTADataFeed = 'http://datamine.mta.info/mta_esi.php?key=bdac84d39bd868f55d63fabb8686d924&feed_id=51'
r = getRequest(MTADataFeed)
feed = createFeed(r)
dictFeed = convertToDictFeed(r, feed)
# pprintDictFeed(dictFeed)
trainScheduleDataList, trainCurrentStopDataList = createTrainDataLists(dictFeed)
# printTrainDataLists(trainScheduleDataList, trainCurrentStopDataList)
# Available Train Schedule Data:
trainScheduleIDList, trainScheduleTripUpdateList = getIDsAndTripUpdate(trainScheduleDataList)
trainScheduleTripList, trainScheduleStopTimeUpdateList = getTrainScheduleTripListAndStopTimeUpdateList(trainScheduleTripUpdateList)
trainScheduleTripIDList, trainScheduleTripStartDateList, trainScheduleTrainRouteIDList = obtainTrainScheduleTripListData(trainScheduleTripList)
trainScheduleArrivalTimeList, trainScheduleDepartureTimeList, trainScheduleStopIDList, trainScheduleRelationshipList = obtainTrainScheduleStopTimeUpdateListData(trainScheduleStopTimeUpdateList)
# Available Current Stop Data:
trainCurrentStopIDList, trainCurrentStopTripIDList, trainCurrentStopStartDateList, trainCurrentStopRouteIDList, trainCurrentStopCurrentStopSequenceList, trainCurrentStopCurrentStatusList, trainCurrentStopTimestampList = processTrainCurrentStopDataList(trainCurrentStopDataList)
count = countVehicles(feed)
# print('Checking IDs in main(): ')
# print('Checking trainScheduleIDList: ')
# print('len(trainScheduleIDList) = ' + str(len(trainScheduleIDList)))
# for item in trainScheduleIDList:
# print(item)
# print('Checking trainCurrentStopIDList: ')
# print('len(trainCurrentStopIDList) = ' + str(len(trainCurrentStopIDList)))
# for item in trainCurrentStopIDList:
# print(item)
# Convert Schedule Data To Panda:
schedule_df = convertToPanda1(trainScheduleArrivalTimeList, trainScheduleDepartureTimeList, trainScheduleStopIDList, trainScheduleRelationshipList)
# Convert Current Stop Data To Panda:
currentstop_df1, currentstop_df2 = convertToPanda2(trainScheduleIDList, trainScheduleTripIDList, trainScheduleTripStartDateList, trainScheduleTrainRouteIDList, trainCurrentStopIDList, trainCurrentStopTripIDList, trainCurrentStopStartDateList, trainCurrentStopRouteIDList, trainCurrentStopCurrentStopSequenceList, trainCurrentStopCurrentStatusList, trainCurrentStopTimestampList)
xlsxWorkbookName = dumpToXLSX(schedule_df, currentstop_df1, currentstop_df2)
if __name__ == '__main__':
main()
You can’t perform that action at this time.