Running the Graple Program

Running Simulations using Graple

Running a simulation using HT-Condor, IPOP, and Python is a reasonably straight forward endeavor.

Prerequisites:

1. An x86 PC running Windows 7 or Windows Server 2008 R2.
2. IPOP VPN
3. HT-Condor
4. Python 2.7
5. GRAPLE Program

Sim – a single instance or run of a simulation.
Job – one or more simulations archived together and executed serially as a single batch operation.

Simulation format and layout

To leverage the functionality of Graple, you will need to organize the files and directories of your simulation in the following manner. Create a top level folder, e.g., “SimRoot”, which will contain all the files pertaining to your experiment. Running Graple from this location will automatically create the necessary directories -- Condor, GLM, Graple, Results, Scripts, Sims. These directories will contain the following files:

• Condor – the Condor scripts used to schedule the jobs. These scripts are automatically generated only when needed.
• GLM – the GLM executable and libraries for running the simulations. You are required to put the appropriate binaries for a fully functional setup here. This includes the GLM executable and its associated dynamically linked libraries (DLL).
• Graple – the core program file for job batching which optimizes workload and simplifies usage. This file is available for download here.
• Scripts – One or more R scripts used for post-processing the GLM output. These are your scripts that you can use to post process the GLM output.
• Sims – contains sub-directories of simulation folders. Each one is a self-contained simulation to be run and should be named Simxxx (where xxx is a monotonic increasing numerical suffix). Typically each Simxxx directory contains the NML and CSV files associated with each individual experiment.
• Results - The output from your experiments will be placed here.

There are also two additional files, SubmitGrapleBatch.py and ProcessGrapleBatchOutputs.py, in the SimRoot folder that further simply submitting and collecting results.

IMAGE Placeholder

Workflow

There are 3 phases of Graple’s workflow – Simulation Submission, Execution, and Results Collation.

###Simulation Submission This is the mode for Graple when you invoke SubmitGrapleBatch. A series of ZIP archives are created where each one represents a batch job that is scheduled with Condor. Each archive contains the GLM, Scripts, and one or more Sims/Simxxx folders. On the Condor execute node all files in the archive will be unpacked to a single unique directory and the simulation executed.

###Simulation Execution This is automatically invoked on the Condor execute node. For each simulation Graple will run GLM and place its output in a Results sub-directory. It will then invoke the top level R script with the Results directory path. The output of R should be placed in the same folder. Upon completion of all the simulaltions, Graple will archive the all the outputs that were run in the job. The Results archive will then be transferred to the host that initiated the job.

Results Collation

When your simulation is complete -- this can be verified with condor_status or condor_q -- invoking ProcessGrapleBatchOutputs.py will unpack the results archives for all the jobs that have been completed. The results are copied into the SimsRoot/Results/Simxxx/results directory.

##Usage

1. Download Graple setup archive
2. Extract the contents of the archive to a location of your choice, eg., _c:_ .
3. Copy your files into GLM, Scripts & Sims folders.
4. From the SimRoot folder, invoke "python SubmitGrapleBatch.py"
5. Check for job completion using condor_status
6. From the SimRoot folder, run "python ProcessGrapleBatchOutputs.py"
   Note that you can also double click the py files from within windows explorer.