Output staging initial implementation

Output staging initial implementation

Our initial implementation of Output staging vi SRM is present in ifdhc version v1_2_1 and later, and uses an environment variable IFDH_STAGE_VIA which is either:

• The URI of an srm: service
• an environment variable name (i.e. $OSG_SITE_WRITE) which should map to an srm: URI.

So to use this facility, you should just need to pass in this environment variable when you submit a job that uses ifdh cp to copy files out; i.e.

*@
export IFDH_STAGE_VIA='$OSG_SITE_WRITE' # note quotes
jobsub -e IFDH_STAGE_VIA ...
@*

Note that any ifhc that comes along with IFDH_STAGE_VIA set will try to run the copyback script to stage the data back to Fermilab.
Note also that IFDH_STAGE_VIA is not curently compatable with gridftp specific url usage --- only unlabelled, srm:, and file: locations are currently supported.

If you are submitting to a site that doesn't have a local SRM storage element, you can specify a specific one, rather than telling it to look in an environment variable. For example to stage via volatile DCACHE here at Fermilab:

*@
export IFDH_STAGE_VIA='srm://fndca1.fnal.gov:8443/srm/managerv2?SFN=/pnfs/fnal.gov/usr/fermigrid/volatile'
jobsub -e IFDH_STAGE_VIA ...
@*

For other sites:

SMU

p(. export IFDH_STAGE_VIA='srm://mfosgse.ehpc.smu.edu:8443/srm/v2/server?SFN=/data/srm'

Nebraska

p(. export IFDH_STAGE_VIA='srm://dcache07.unl.edu:8443/srm/v2/server?SFN=/mnt/hadoop/user'

Official SE list for "nova" group

UCSDT2: (UCSD)

srm-ping srm://bsrm-3.t2.ucsd.edu:8443/srm/v2/server
mapped to
gumsIDMapped=novaana
SURL: srm://bsrm-3.t2.ucsd.edu:8443/srm/v2/server?SFN=/hadoop/nova
GOC ticket has been opened

GLOW: (Wisconsin)
srm://cmssrm.hep.wisc.edu:8443/srm/v2/server
gumsIDMapped=osg_fermilab
SURL=srm://cmssrm.hep.wisc.edu:8443/srm/v2/server?SFN=/hdfs/osg/vo/
GOC ticket has been opened
srm://cmssrm.hep.wisc.edu:8443/srm/v2/server?SFN=/hdfs/osg/vo/nova/

Nebraska:
srm://dcache07.unl.edu:8443/srm/v2/server
gumsIDMapped=nova
SURL: srm://dcache07.unl.edu:8443?SFN=/mnt/hadoop/user/nova
working!

SMU:
srm://mfosgse.ehpc.smu.edu:8443/srm/v2/server?SFN=/data/srm

gumsIDMapped=novaana

SURL:srm://smuosgse.hpc.smu.edu:8443/srm/v2/server?SFN=/data/srm/nova
working!