Allow the usage of a secondary dataset. #4934
Conversation
Simple n² algorithm to perform the matching based on file lumi sections. Took about 30 seconds doing matching on 2012D data (USER + RAW) of a dataset I found in DBS. Resolves dmwm#4861.
* Make changes compatible with old REST interface. * Only apply `use_parent` where needed. * Log before and after matching computation.
Allow the usage of a secondary dataset.
| #Job Type Section | ||
| config.section_("JobType") | ||
| config.JobType.pluginName = 'Analysis' | ||
| config.JobType.psetName = 'psets/pset_parent_analysis.py' |
There was a problem hiding this comment.
Is this the same as psets/pset_use_parent.py ? this file which is in config does not exist.
There was a problem hiding this comment.
Ah, sorry! Yes, it should be!
There was a problem hiding this comment.
Did this get recently renamed? I had copied the config for parent usage and changed it? I think I might have overlooked file name changes when rebasing…
There was a problem hiding this comment.
Weird. Maybe I just had that file left over then from something else. Sorry again!
There was a problem hiding this comment.
Hi, I tried to test it but jobs run into a CMSSW error. Do I need CMSSW version greater than CMSSW_7_0_5 in order to run it?
==== CMSSW Stack Execution FAILED at Wed Sep 2 12:40:15 2015 ====
======== CMSSW OUTPUT STARTING ========
== CMSSW: Beginning CMSSW wrapper script
== CMSSW: slc6_amd64_gcc481 scramv1 CMSSW
== CMSSW: Performing SCRAM setup...
== CMSSW: Completed SCRAM setup
== CMSSW: Retrieving SCRAM project...
== CMSSW: Untarring /home/grid/cmsplt17/home_cream_362800243/CREAM362800243/glide_NJ7wT3/execute/dir_3952/sandbox.tar.gz
== CMSSW: Completed SCRAM project
== CMSSW: Executing CMSSW
== CMSSW: cmsRun -j FrameworkJobReport.xml PSet.py
== CMSSW: 02-Sep-2015 14:39:52 CEST Initiating request to open file file:/gpfs/ddn/srm/cms/store/data/Run2015B/SingleMu/MINIAOD/17Jul2015-v1/30000/CE954B54-BE2E-E511-8C94-0026189438F5.root
== CMSSW: 02-Sep-2015 14:39:54 CEST Fallback request to file root://xrootd-cms.infn.it:1194//store/data/Run2015B/SingleMu/MINIAOD/17Jul2015-v1/30000/CE954B54-BE2E-E511-8C94-0026189438F5.root
== CMSSW: 150902 14:40:14 4279 Xrd: CheckErrorStatus: Server [cmssrmdisk.fnal.gov:1095] declared: Request 3018 not supported(error code: 3013)
== CMSSW: ----- Begin Fatal Exception 02-Sep-2015 14:40:15 CEST-----------------------
== CMSSW: An exception of category 'FallbackFileOpenError' occurred while
== CMSSW: [0] Constructing the EventProcessor
== CMSSW: [1] Constructing input source of type PoolSource
== CMSSW: [2] Calling RootInputFileSequence::initFile()
== CMSSW: Additional Info:
== CMSSW: [a] Input file file:/gpfs/ddn/srm/cms/store/data/Run2015B/SingleMu/MINIAOD/17Jul2015-v1/30000/CE954B54-BE2E-E511-8C94-0026189438F5.root could not be opened.
== CMSSW: Fallback Input file root://xrootd-cms.infn.it:1194//store/data/Run2015B/SingleMu/MINIAOD/17Jul2015-v1/30000/CE954B54-BE2E-E511-8C94-0026189438F5.root also could not be opened.
== CMSSW: [b] Fatal Root Error: @sub=TStreamerInfo::BuildCheck
== CMSSW:
== CMSSW: The StreamerInfo for version 1 of class TObject read from the file root://xrootd-cms.infn.it:1194//store/data/Run2015B/SingleMu/MINIAOD/17Jul2015-v1/30000/CE954B54-BE2E-E511-8C94-0026189438F5.root
== CMSSW: has a different checksum than the previously loaded StreamerInfo.
== CMSSW: Reading objects of type TObject from the file root://xrootd-cms.infn.it:1194//store/data/Run2015B/SingleMu/MINIAOD/17Jul2015-v1/30000/CE954B54-BE2E-E511-8C94-0026189438F5.root
== CMSSW: (and potentially other files) might not work correctly.
== CMSSW: Most likely the version number of the class was not properly
== CMSSW: updated [See ClassDef(TObject,1)].
== CMSSW:
== CMSSW: ----- End Fatal Exception -------------------------------------------------
== CMSSW: Complete
== CMSSW: process id is 4279 status is 92
======== CMSSW OUTPUT FINSHING ========
There was a problem hiding this comment.
Ah, yes. I think I've used 75 before.
Simple n² algorithm to perform the matching based on file lumi sections.
Took about 30 seconds doing matching on 2012D data (USER + RAW) of a
dataset I found in DBS.
Note that this could still easily be expanded to include an arbitrary number of datasets, although computational cost will increase…
Resolves #4861.