🧹 Chore: Are we using all of the skim tables we create in mode choice?

[DavidOry]

The majority of model runtime is devoted to transit assignment and skimming. One way to reduce the burden is to not generate skim tables that are not used by the downstream model. For example, do we need "drive-transit-walk" skims for the evening period? Do we need technology-specific IVT tables for each of the skims? By reducing skims and/or skim tables, we can improve runtime.

Progress:

• Sufficiently defined
• Approach decided
• Implemented

Considerations

• Some skim tables may not be necessary, but may be created to give flexibility in calibration. For example, it's easy to bump up heavy rail trips if we have access to a heavy rail travel time skim that identifies which interchanges are using heavy rail.

fyi @Ennazus. Adding this to the technical debt project.

[DavidOry]

@Ennazus FYI moving this to Task Order 6.

As discussed on Slack, we are devoting a lot of our computing time to writing out transit skims and, per the above issue, it may be that this is due, in part, to writing out skims we do not need.

@yueshuaing Looked through the log file and it seems it is taking, per time period, about 45 minutes to create the skims and 30 minutes to write them out. I'm attaching a spreadsheet she made to log the estimates.

transit-skim-runtime-log.xlsx

I am thinking we should do three things:

1. Check to see if we are currently using any of the in-vehicle time by mode tables (in the mode choice models or the acceptance criteria or anywhere else). We created them to assist in calibration, but we may not be using them (yet). If not, we can add a switch to the model_config to not create these. This step is taking about 3 minutes per time period per path (per global iteration), but it varies and can be as much as 10 minutes.
2. Talk to @inrokevin to see if parallel processing would be useful. We had previously talked about parallel processing transit assignment (🚀 Feature: Serial or Parallel Processing for Transit Assignment #182), but given that transit assignment is maxing out our compute, it may make more sense to parallel process the skimming. While this may not help the writing the skims out part, it should help with the skim creation part.
3. Go through the code and make low-hanging improvements. For example, it appears we are creating the drive skims for all the paths, even the walk-transit-walk path.

@Ennazus: okay if we proceed on 1. and 3. above? This should take about 2 hours of my time, 2 hours of Ashish's time, and 12 hours of Yue's time. Number 2. is the bigger lift, but would want to get Kevin's feedback on that before we dive in.

cc: @AshishKuls

[Ennazus]

Sounds great to proceed on 1 and 3.

[yueshuaing]

Update on item 1:
The in-vehicle time by mode skims are used by CTRAMP, bestTransitPath utility. We can't skip them from the skimming.

Update on item 3:
I disabled the steps that create drive skims for wlk_trn_wlk mode, walk access skims for prn/knr_trn_wlk, walk egress skims for wlk_trn_pnr/knr. I also skipped exporting several unused skims in CTRAMP or in the summary, including _WAIT, _TRIM, _DTOLL, XBOATIME, _IN_VEHICLE_TIME.
The two modifications above saved around 1hr40mins in transit skimming per iteration. Code changes have been committed to improve_transit_skim branch

[yueshuaing]

Besides the two items above, I also found two possible places for runtime reduction.

1. The transit assignment step currently writes out several assignment outputs (e.g. boardings by line, station to station flow, transfers at station) at the end of each assignment run. Those steps take a significate amount of time ranging from 45min-1hour per time period per iteration. The most time-consuming steps are writing out the station-to-station flow and transfers at station. The Emme tools used, station_to_station_analysis and transfers_at_stops, can take 1-3 mins each time they run.

2. Running transit skimming standalone takes about 3+ hours. However when run as part the model run, i.e. running assignment followed by skimming, the skimming step takes up to 7 hours. Further investigation is need to understand the cause of this discrepancy.

For my time usage, I have already spent about 16 hrs to date and expect to spend another 40 to 60 hrs to (a) disable the station-to-station analysis (except in the last iteration) and (b) test the code to turn the “transfers_at_stops” feature on or off.

[DavidOry]

@Ennazus additional notes on this from conversations with @inrokevin:

1. Right now, the skim tables are first written to the EMME databank and then written to disk as OMX files. On our call last week, we discussed the possibility for avoiding one of these writes. Since we need the OMX files to be consumed by the CT-RAMP Java software (now; and we'll need the OMX files to be consumed by activitysim in the future), the only write we could potentially avoid is the one to the EMME databank. @inrokevin informs me that there is no way to directly write to OMX from skim data stored in memory. The data must be written to the EMME databank first.
2. Right now, we do the transit assignment and write the resulting paths to disk. After each of the five transit assignments are complete, the paths are then analyzed/queried to create the skim tables. On our call last week, we discussed whether it could potentially be more efficient to do an assignment, store the paths in memory, and then go directly to skims. So the cadence would be assignment, skim, assignment, skim, ... rather than assignment, assignment, ..., skim, skim, .... @inrokevin informs me that the paths (strategies in EMME parlance) are always written to disk, so there is no theoretical advantage to doing this. In @yueshuaing's testing, the assignment, skim, assignment, skim cadence appears to take longer than the other cadence, but this may be too to compute issues (i.e., memory needs time to free up?), as @yueshuaing noted above.

[yueshuaing]

Update on transit summaries:
I added the condition 3be4add to only write out station-to-station and transfer-at-station summaries in the last iteration. These two steps can take up to 30 minutes per time period per iteration, so this adjustment significantly reduced the summary runtime —saving approximately 3 hours during one iteration of transit assignment.

A comparison of runtimes before and after the transit skim and summary optimizations is available in the spreadsheet linked below. The spreadsheet also includes a list of skims and summaries that were disabled during testing.

transit-assignment-skim-runtime-log-update.xlsx

(Total time used from the budget so far: 16 + 10 = 26 hours)