Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
A version of Torque (PBS) that runs without root permissions http://www.adaptivecomputing.com/prod…
Fetching latest commit…
Cannot retrieve the latest commit at this time.
|Failed to load latest commit information.|
This file contains information concerning the use of the new job array features in TORQUE 2.5. --- WARNING --- TORQUE 2.5 uses a new format for job arrays. It in not backwards compatible with job arrays from version 2.3 or 2.4. Therefore, it is imperative that the system be drained of any job arrays BEFORE upgrading. Upgrading with job arrays queued or running may cause data loss, crashes, etc, and is not supported. COMMAND UPDATES FOR ARRAYS -------------------------- The commands qalter, qdel, qhold, and qrls now all support TORQUE arrays and will have to be updated. The general command syntax is: command <array_name> [-t array_range] [other command options] The array ranges accepted by -t here are exactly the same as the array ranges that can be specified in qsub. (http://www.clusterresources.com/products/torque/docs/commands/qsub.shtml) SLOT LIMITS -------------------------- It is now possible to limit the number of jobs that can run concurrently in a job array. This is called a slot limit, and the default is unlimited. The slot limit can be set in two ways. The first method can be done at job submission: qsub script.sh -t 0-299%5 This sets the slot limit to 5, meaning only 5 jobs from this array can be running at the same time. The second method can be done on a server wide basis using the server parameter max_slot_limit. Since administrators are more likely to be concerned with limiting arrays than users in many cases the max_slot_limit parameter is a convenient way to set a global policy. If max_slot_limit is not set then the default limit is unlimited. To set max_slot_limit you can use the following queue manager command. qmgr -c 'set server max_slot_limit=10' This means that no array can request a slot limit greater than 10, and any array not requesting a slot limit will receive a slot limit of 10. If a user requests a slot limit greater than 10, the job will be rejected with the message: Requested slot limit is too large, limit is X. In this case, X would be 10. It is recommended that if you are using torque with a scheduler like Moab or Maui that you also set the server parameter moab_array_compatible=true. Setting moab_array_compatible will put all jobs over the slot limit on hold so the scheduler will not try and schedule jobs above the slot limit. JOB ARRAY DEPENDENCIES -------------------------- The following dependencies can now be used for job arrays: afterstartarray afterokarray afternotokarray afteranyarray beforestartarray beforeokarray beforenotokarray beforeanyarray The general syntax is: qsub script.sh -W depend=dependtype:array_name[num_jobs] The suffix [num_jobs] should appear exactly as above, although the number of jobs is optional. If it isn't specified, the dependency will assume that it is the entire array, for example: qsub script.sh -W depend=afterokarray:427 will assume every job in array 427 has to finish successfully for the dependency to be satisfied. The submission: qsub script.sh -W depend=afterokarray:427 means that 5 of the jobs in array 427 have to successfully finish in order for the dependency to be satisfied. NOTE: It is important to remember that the "" is part of the array name. QSTAT FOR JOB ARRAYS -------------------------- Normal qstat output will display a summary of the array instead of displaying the entire array, job for job. qstat -t will expand the output to display the entire array. ARRAY NAMING CONVENTION -------------------------- Arrays are now named with brackets following the array name, for example: dbeer@napali:~/dev/torque/array_changes$ echo sleep 20 | qsub -t 0-299 189.napali Individual jobs in the array are now also noted using square brackets instead of dashes, for example, here is part of the output of qstat -t for the above array: 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch 189.napali STDIN dbeer 0 Q batch