Running Scenarios

Running SWIM Scenarios

This section contains the following sub-sections:

Creating a New Scenario

To create a new model scenario:

Pull recent changes from the git repository.
Make a copy of the scenario folder, and name it something meaningful to the Scenario, for example scenario_name.
Ensure the model files in tlumip/root/model/ are unpacked. See the README file in census, faf, and lib folder within tlumip/root/model/ for further instructions.
Make all scenario specific changes (editing network and land use in Visum, ALD and AA inputs in parameters,...)
Ensure the tsteps.csv file is configured to run the modules and years that are intended for the new scenario.
Build the run_model files by running build_run.bat, this creates run_model.bat
Run the model using the run_model.bat that the build_run.bat creates

Those steps allow a user to setup a new model scenario and those steps assume that the model will run from current calibration start year(s), which is 2009-10.

Important Setup Note - tXX, where XX is a two digit year from t0 (t0 is 1990, t20 = 2010), is a hard coded naming scheme in SWIM. SWIM sets up a number of folders and makes assumptions around where to store information based on tXX. There are run issues (run fails) when a scenario is setup with tXX in the scenario. Example, recently the scenario name "Ref26_Sept21" was used. This name meant that we were doing a test of the Reference 2.6 repo in September 2021. However, Ref26_Sept21 has a t21 in it. So the model run crashed. The solution - name the scenario "Ref_Sep21". This error can be hard to track and can cause a lot of spinning and down time - please read and remember this note.

Start a Scenario from Mid Year

Given how historic time (years) now exists (between 2009 and the present), a new run type has been developed allowing scenarios to pivot off of a historic run series (2009 to the current year). A tsteps file now exist in the model\config folder\tsteps_Historic.csv to do a historic run (2009 through 2019). Before startng future scenarios from a mid year, the user should have a historic run completed to have outputs till the validation year.

Next, the user would follow these sequential steps:

Create a new scenario using steps 1-6 in Creating a New Scenario
Copy t29FileCopy.R to root\scenario and update the script for "dataDir" (historic runs with outputs up to the validation year) and "validationYear" (ex. 29)
Open a command prompt in the scenario folder and run the following command: "..\model\lib\R-3.3.2\bin\x64\Rscript.exe" "t29FileCopy.R". This will populate the new scenarios output folder with the required historic files to kick off the future years (currently 2020 and beyond).
Copy RunFuture.R to root\scenario and update the script for "StartPoint" (the year following the validation year, ex. "Starting SI in year 30" if validationYear is 29)
Open a command prompt in the scenario folder and run the following command: "..\model\lib\R-3.3.2\bin\x64\Rscript.exe" "RunFuture.R"
Run run_model_Future.bat

Restarting a failed model run

To restart a failed model run, you

Copy model/code/model_runner/model_runner_batch.bat into the scenario_name directory.
Delete all commands after the java startup prior to the fail point.
Run the edited script.

For example, if you wanted to start the model in TA for year 23, your edited model_runner_batch.bat file would look like this:

@ECHO OFF
echo. 2>"E:\tlumip\root\transit_fix\model_report.txt"
cmd /C ""E:\tlumip\root\transit_fix\model\code\report.bat" *****Model run started*****>>"E:\tlumip\root\transit_fix\model_report.txt" 2>&1"
cmd /C ""E:\tlumip\root\transit_fix\model\code\report.bat" Starting file monitor>>"E:\tlumip\root\transit_fix\model_report.txt" 2>&1"
start "" "E:\tlumip\root\model\lib\java7\jre\bin\java.exe" -cp "E:\tlumip\root\transit_fix\model\code\tlumip.jar;E:\tlumip\root\transit_fix\model\code\xmlrpc-2.0.jar;E:\tlumip\root\transit_fix\model\code\commons-codec-1.3.jar;E:\tlumip\root\transit_fix\model\code;E:\tlumip\root\transit_fix\model\config" "-Dlog4j.configuration=info_log4j_fileMonitor.xml" -server com.pb.common.daf.admin.FileMonitor "E:/tlumip/root/transit_fix/model/config/commandFile.txt" "E:/tlumip/root/transit_fix/model/config/startnode0.properties"
cmd /C "ping 127.0.0.1 -n 31 > NUL"
IF %ERRORLEVEL% NEQ 0 GOTO MODEL_ERROR
# deleted lines #
cmd /C ""E:\tlumip\root\transit_fix\model\code\report.bat" Starting TA in year 23>>"E:\tlumip\root\transit_fix\model_report.txt" 2>&1"
"E:\tlumip\root\model\lib\java7\jre\bin\java.exe" -cp "E:\tlumip\root\transit_fix\model\code\tlumip.jar;E:\tlumip\root\transit_fix\model\code\xmlrpc-2.0.jar;E:\tlumip\root\transit_fix\model\code\commons-codec-1.3.jar;E:\tlumip\root\transit_fix\model\code;E:\tlumip\root\transit_fix\model\config" -Xmx3000m "-Dlog4j.configuration=info_log4j.xml" -server com.pb.tlumip.ao.ModelEntry NED "property_file=E:/tlumip/root/transit_fix/outputs/t22/ned.properties"
IF %ERRORLEVEL% NEQ 0 GOTO MODEL_ERROR

NOTE: If the run fails using the edited model_runner_batch.bat then you should restart the run using the default method. Here are the steps:

Edit tsteps.csv by removing '1' from the steps that were already run until the failed stage
Build a run using build_run.bat
Restart the run using run_model.bat

Adjusting Scenario Attributes

Scenario-Specific Modifications

If there are code updates or other structural changes to the model, then the files in the scenario_name/model/ directories may need to be changed. In general, the files which are changed to make a scenario unique are contained in the scenario_name/inputs/ directory. The scenario_name/inputs/parameters/ and scenario_name/inputs/t[year]/ directories contain various files that can be used to customize a scenario, and the descriptions of these files can be found in each of the specific module sections, for instance input files for NED are described in NED. Below is a list of general model inputs locations that are not module specific. Each of the folders below contains a readme with more specific instructions on all of the individual files that are contained in each folder:

The /outputs/ folder also has inputs (ironically) for t18 and t19. These inputs feed AA, PT, and CT and are discussed in more detail in those sections.

Scenario Properties

SWIM runs through time. There are some inputs that are constant through-out the model run, and some that are only used for starting years. The globalTemplate.properties file sets a large number of pointers and single value parameters for all of the modules in SWIM. In addition to those, more static inputs, there are also inputs that need to change over the course of a ~30 year SWIM run. For example, the network that is modeled might change in future years as new projects are added or removed for future analysis years. Another example (that can be seen in the git repository) is where the first year conditions (year t19) are slightly different than all other future years. The globalTemplateUpdate.properties file is the way a user can specify specific inputs that change in future years.

Global Template Update Properties (/inputs/t*)

As stated above, the globalTemplate.properties file sets a large number of pointers and single value parameters for all of the modules in SWIM. To change any of those pointers or parameters, a user creates a new t* folder and then creates a globalTemplateUpdate.properties text file in that new folder to update specific parameters and file points in the properties files used by the model (it's important to know that while the model runs, module specific properties files are created in the outputs folder for each year, example - scenario_name/outputs/t20/ned.properties). Therefore, the properties that the model ultimately uses for any given year are a mix of what exists in /inputs/globalTemplate.properties and all the changes that are represented in the globalTemplateUpdate.properties files up through year x-1.

As an example from the git repository, if the scenario needed different parameters for the initial year, t19 (t19 is 1990 + 19 = 2009), from the following modeled years (which is required for SWIM runs), then the scenario would have an scenario_name/inputs/t19 folder with a globalTemplateUpdate.properties specifying what changes past t19. Again, taking the git repository as an example, the initial model year (t19) runs with the properties as specified in globalTemplate.properties. However, when t20 starts to run, the model runner process looks for t20-1 folders (t19 and below) and adds those changes to the globalTemplate.properties in sequential order (t19, t20...). So for the git repository code, t19 uses the properties as specified in globalTemplate.properties. The run for t20 (2010) uses all the properties in globalTemplate.properties, but with the edits found in t19/globalTemplateUpdate.properties, see following code block:

# Changes to AA running past the bootstrap year.
# The Global properties file sets these values to FALSE because the files do not exist in the intial year.
# This changes the settings so that in future years, AA will pull the files that NED and SPG create.
aa.readHouseholdsByHHCategory=true
aa.updateImportsAndExports=true

#copy over the new t19 activity locations to t18 (can only do this after AA was run in t19, so at SI in t20)
ald.copyToPreviousYear = true

And finally, year t21 (2011) and beyond use globalTemplate.properties, plus changes in t19/globalTemplateUpdate.properties, plus changes from t20/globalTemplateUpdate.properties](https://github.com/tlumip/tlumip/blob/master/root/scenario/inputs/t20/globalTemplateUpdate.properties), see t20 edit in following code block (notice that this change overwrites a t19 setting from above:

#Only copy t19 skims to t18 (so turn it off for all other years)
ald.copyToPreviousYear = false

Another example of an input the user will likely want to update in a scenario is the zoning attribution or network configuration housed in the Visum version file /inputs/t0. As an example, the git repository will soon have a 2016 highway projects network to be used for years 2016 and beyond. When this is added the repo will get a t25 folder, and a new swimNetworkAttributes.ver file will be added to that folder, which will contain the 2016 network. In this case, because the pointer for this version file is defined in the inputs/globalTemplate.properties as:

ta.base.version.file = @ROOT.DIR@/@SCENARIO.NAME@/@SCENARIO.INPUTS@/t@CURRENT.INTERVAL@/swimNetworkAttributes.ver

The configuration could just provide a new version file in inputs/t25 with the same file name and the code will find it and point to it (we hope, haven't fully tested as of yet, Feb 2016). It would likely be better practice to change the version file name to something like swimNetworkAttributes_2016.ver. In which case the git repository would also have a t25/globalTemplateUpdate.properties file with the single line (plus comments):

ta.base.version.file = @ROOT.DIR@/@SCENARIO.NAME@/@SCENARIO.INPUTS@/t@CURRENT.INTERVAL@/swimNetworkAttributes_2016.ver

Makes sense? Great. Now go get her Ray!

Scenario Setup Examples

The SWIM acceptance testing resulted in a handful of new scenarios that can be adapted for other studies. These scenarios are described on their own page:

Extra user guide - haven't updated to the wiki yet

Also in the scenario’s /inputs/ directory are properties files which are used to specify various things about a model run. The main model properties are specified in the /inputs/globalTemplate.properties file as described in SWIM_properties.xlsx. This file is “templated” with keywords that are replaced during the model build process and the “detemplified” file is saved in various (module-specific) properties files in the scenario’s /outputs/t[year]/ folders. The available template keywords are as follows:

@ROOT.DIR@ - the model install directory
@CURRENT.INTERVAL@ - the current “t” year
@SCENARIO.NAME@ - the scenario name
@SCENARIO.INPUTS@ - the scenario’s inputs directory (relative to the scenario directory)
@SCENARIO.OUTPUTS@ - the scenario’s outputs directory (relative to the scenario directory)
@[MODULE].LAST.RUN@ - the “t” year that a module last ran (or was initialized for bootstrapping purposes).
@AA.PRIOR.RUN@ - the second-to-last “t” year that the AA module last ran (or was initialized for bootstrapping purposes).
@SL.MODE@ - this will detemplify to the name of the SL mode that will be run.
@PT.LOGSUMS@ - if PT logsums will be calculated, then this will detemplify to “true”; otherwise it will detemplify to “false”
@PT.LDT@ - if PT LDT will be calculated, then this will detemplify to “true”; otherwise it will detemplify to “false”
@PT.SDT@ - if PT SDT will be calculated, then this will detemplify to “true”; otherwise it will detemplify to “false”
@VIZ_FINAL_YEAR@ - All the VIZ databases compiled before this year including the final year will be combined.

The available module names are:

SI – Swim Inputs
NED - New regional Economics and Demographics
ALD - Aggregate Land Development
SPG1 - First phase of the Synthetic Population Generator
POPSIMSPG1 - First phase of the PopulationSim Synthetic Population Generator
AA – Activity Allocation (aka PECAS)
SPG2 - Second phase of the Synthetic Population Generator
POPSIMSPG2 - Second phase of the PopulationSim Synthetic Population Generator
PT - Personal Transport
CT - Commercial Transport
TA – Traffic (highway) Assignment
TR – Transit Assignment

Additionally, it is sometimes necessary to update a property file at certain points in a model run’s multi-year sequence. To allow for this, property file “updates” may be specified in /inputs/t[year]/globalTemplateUpdate.properties files. In any model run year starting at t[year], the properties in this update file supersede all of those that came before it.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly