-
Notifications
You must be signed in to change notification settings - Fork 2
Overview
By connecting the scientific community with data and model resources, the GLEON Research and PRAGMA Lake Expedition (GRAPLE) enables advanced simulation of lake hydrodynamics and water quality to address research questions. One activity in the expedition is the deployment of an IPOP overlay virtual network and the HTCondor batch job scheduler across participating GLEON/PRAGMA institutions to enable aggregation and sharing of computational resources and models by a distributed group of collaborators.
Participants in the expedition are able to easily join a HTCondor high-throughput computing resource pool, with their own local computers (desktops, laptops, servers) by using the IPOP virtual network overlay. Once connected to the HTCondor pool, users can then run batches of simulations (such as parametric sweep runs of the GLM model) in an efficient manner. This document aims to provide an overview of HTCondor and the IPOP GroupVPN virtual network for first-time users of the GRAPLE infrastructure.
Figure 1. Overall architecture of HTCondor pool for GRAPLE. The pool consists of distributed resources connected by the IPOP GroupVPN overlay, and allows users to run batches of simulation jobs (e.g. GLM) across the distributed resources.
A HTCondor pool consisting of a server and nodes runs on an IP-over-P2P (IPOP) overlay network connecting up a server and all nodes securely. An example architecture of HTCondor pool is shown in Figure 1. A server is located in the University of Florida and nodes are distributed all around world: at Virginia Tech, University of Wisconsin, Center of Ecological Research in Japan, and University of Technology in Sydney. In each HTCondor node, a GroupVPN controller and a HTCondor program are installed. To run GLM simulations, GLM software is also installed in nodes.
- Preparation step To run a GLM simulation job through HTCondor, each node must install GroupVPN controller, HTCondor program, and GLM program before proceeding. a. GroupVPN installation : https://github.com/GRAPLE/documentation/wiki/GroupVPN-Installation b. HTCondor installation : https://github.com/GRAPLE/documentation/wiki/HTCondor-Installation c. GLM for Windows installation : Go to the following link and enter your email address. Then you can download GLM software. http://aed.see.uwa.edu.au/research/models/GLM/Pages/download_win.php
- Job submission step a. Creation of a job description file : Please refer an example job description file ‘condor_mendota.config’ b. Job submission to the HTCondor server i) Single job : Please refer the example submission for single simulation of lake Mendota described in https://github.com/GRAPLE/documentation/wiki/Lake-Mendota-simulation ii) Parametric sweep job c. Verification of results
a) A job submission to the server from UTS
b) node4 receives a list of available nodes, UW and You
c) UTS sends input files to UW and You
d) UTS receives GLM simulation results from UW and You
Figure 2. Procedures for example job submission and results check a) A node at UTS submits a job description file to the server at UF, b) The server runs its matchmaker to find available nodes and then sends a list of nodes, c) After receiving a list, the UTS node sends job requests to two nodes, the UW node and you, d) Each node runs the requested GLM simulation and returns a result to the UTS node.