Supported Adaptors & Resources

drelu edited this page Feb 19, 2013 · 5 revisions

service_url

The service_url is part of the pilot_compute_description. The service URL communicates what type of queueing system or middleware you want to use and where it is. localhost can be changed to a machine-specific URL, for example: sge://lonestar.tacc.utexas.edu. The following table explains the supported middleware on XSEDE and FutureGrid. Note: You WILL have to edit the examples for your personal middleware or queueing system.

Supported Adaptors Description XSEDE Resources
(Examples)
fork Submit jobs only on localhost head node. Example usage: fork://localhost Any
SSH Allows to submit jobs to a remote host via SSH. Password-less login to target machine is required. Example usage: ssh://eric1.loni.org Any
SGE(+SSH/GSISSH) Interfaces with a Sun Grid Engine (SGE) scheduler locally or remotely via SSH. Password-less login to target machine is required. Example usage:
Local: sge://localhost
Remote (over SSH): sge+ssh://lonestar.tacc.utexas.edu
Remote (over GSISSH): sge+gsissh://lonestar.tacc.utexas.edu
Lonestar
Ranger
Torque/PBS(+SSH/GSISSH) Submit jobs using gsissh. Globus certificates are required. Initiate grid proxy (myproxy-logon) before executing the BigJob application. Example usage of URL:
Local: pbs://localhost
SSH:pbs+ssh://trestles.sdsc.edu
GSISSH: xt5torque+gsissh://gsissh.kraken.nics.xsede.org
Trestles
Kraken
SLURM+SSH Submit jobs using SLURM and SSH: slurm+ssh://login1.stampede.tacc.utexas.edu Stampede

If jobs are submitted via GSISSH, a valid Globus certificates is required. Initiate grid proxy (myproxy-logon) before executing the BigJob application. Please find the GSISSH resource URLs of XSEDE machines at https://www.xsede.org/wwwteragrid/archive/web/user-support/gram-gatekeepers-gateway.html

Required Attributes of Pilot Description:

The attributes required in the PilotComputeDescription depend on the respective backends.

Mandatory Attributes:

  • service_url
  • number_of_processes

Most production infrastructures additionally require:

  • processes_per_node (required for PBS, Torque, SGE)
  • queue
  • project
  • walltime

Stampede Example PilotComputeDescription:

pilot_compute_description = {
                         "service_url": 'slurm+ssh://login1.stampede.tacc.utexas.edu',
                         "number_of_processes": 32,                             
                         "queue":"normal", 
                         "project":"TG-MCBXXXXXX", # if None default allocation is used 
                         "walltime":10,
                         "working_directory": os.getcwd()
                        }