Crab publish for several outputs in the same jobs

[ericvaandering]

Original Savannah ticket 11231 reported by None on Tue Sep 15 07:56:26 2009.

Dear crab experts,

I was running a test skim workflow on the CAF on the CRAFT09 Cosmics dataset. Every skim jobs has three pooloutput modules, i.e. three edm output files so to save time on reading events. As far as I understand there is no way currently do directly publish multiple outputs on the local DBS. This might be an use case later with group/user skims at the Tier2. Is it something that can be implemented?
Best,
l.

[ericvaandering]

Comment by gowdy on Tue Sep 15 07:56:26 2009

This item has been reassigned from the project CMS Computing Infrastructure Support task tracker to your tracker.

The original report is still available at task #11226

Following are the information included in the original report:

• [field #0] Item ID: 11226
• [field Add tar balls to repo #1] Group ID: 452
• [field Add binaries to externals/ directory #2] Open/Closed: Open
• [field Lmitiation due to the Input sandbox #3] Severity: 3 - Normal
• [field change users LFN namespace, add check on user datasetpath #4] Privacy: Public
• [field support for SRMv2 #9] Category: None
• [field new client-server communication system #10] Submitted by: malgeri
• [field development of a set of API for the dedicate storage interaction #11] Assigned to: None
• [field re-deign of submission component structure #12] Submitted on: 2009-09-15 12:09
• [field CRAB-SA/Client and BossLite integration #13] Summary: Crab publish for several outputs in the same jobs
• [field development of condor_g scheduler script for BossLite #14] Original Submission: Dear crab experts,

I was running a test skim workflow on the CAF on the CRAFT09 Cosmics dataset. Every skim jobs has three pooloutput modules, i.e. three edm output files so to save time on reading events. As far as I understand there is no way currently do directly publish multiple outputs on the local DBS. This might be an use case later with group/user skims at the Tier2. Is it something that can be implemented?
Best,
l.

• [field Improve the Error Handler making the jobs resubission more smart #16] Item Group: None
• [field add the merge support for analysis jobs #17] Status: None
• [field implementation of a "localcopy" feature to take output file from a SE to the user local area #18] Component Version: None
• [field CRAB wrapper cleaning and reorganization #19] Operating System: None
• [field LSF (CAF ) script for bosslite #20] Reproducibility: None
• [field implement a new feature which allow an easy report about submitted jobs/task #21] Size (loc): None
• [field submission to the CAF using the server #22] Fixed Release: None
• [field check how to use (if needed) the PA job queue when the server submit directly to condor_g #23] Planned Release: None
• [field test #24] Effort: 0.00
• [field GetOutput Migration from Boss to bosslite #28] Priority: 5 - Normal
• [field JobTracking component migration to bosslite #29] Should Start On: 2009-09-15 02:00
• [field implement task flavor management in command menager component #30] Should be Finished on: 2009-09-15 02:00
• [field support to additional requirements management #31] Percent Complete: 0%
• [field test of: new support for py pset @ WN level and api to extract hnUserName usage #33] Release: None
• [field Abilitate PostMortem for the server #58] Custom Select Box Add tar balls to repo #1: None
• [field crab -report #59] Custom Select Box Add binaries to externals/ directory #2: None
• [field Enable the bulk resubmissions #60] Custom Select Box Lmitiation due to the Input sandbox #3: None
• [field improve get output status and flag handling in the server #61] Custom Select Box change users LFN namespace, add check on user datasetpath #4: None
• [field CommandManager support for allowed users and for accepted client versions #62] Custom Select Box extract output file name from pset #5: None
• [field [BUG] Logging info being retrieved also for not aborted/killed joba #63] Custom Select Box enable the "global" crab.cfg support #6: None
• [field [BUG] Problem on JobHandling.py: receiving jobid instead of object or viceversa #64] Custom Select Box "multicrab" support implementation #7: None
• [field [BUG] Status updates on resubmissions #65] Custom Select Box Test CRAB with the new LCG-UI #8: None
• [field GetOutput timeout for glite #66] Custom Select Box support for SRMv2 #9: None
• [field CMS Naming B/W listing (aka Brian patch) #67] Custom Select Box new client-server communication system #10: None

[ericvaandering]

Closed by fanzago on Thu Oct 15 10:52:26 2009

[ericvaandering]

Comment by fanzago on Thu Oct 15 10:52:26 2009

I close this task. I have opened the savannah "support"
110416. Please follow this ticket and add details.