Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DQMRootOutputModule: implement support for multi-process cmsRun #2190

Merged
merged 1 commit into from Feb 6, 2014
Merged

DQMRootOutputModule: implement support for multi-process cmsRun #2190

merged 1 commit into from Feb 6, 2014

Conversation

fwyzard
Copy link
Contributor

@fwyzard fwyzard commented Jan 27, 2014

implement postForkReacquireResources(...) for DQMRootOutputModule, adding _NNN to the file name along the same logic used by the PoolOutputModule.

@fwyzard fwyzard mentioned this pull request Jan 27, 2014
Merged
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @fwyzard (Andrea Bocci) for CMSSW_7_0_X.

DQMRootOutputModule: implement support for multi-process cmsRun

It involves the following packages:

DQMServices/FwkIO

@ojeda, @danduggan, @rovere, @cmsbuild, @nclopezo, @deguio, @Degano can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
@ktf you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

@cmsbuild
Copy link
Contributor

-1
When I ran the RelVals I found an error in the following worklfows:
1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVD.log
----- Begin Fatal Exception 28-Jan-2014 14:58:22 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log
----- Begin Fatal Exception 28-Jan-2014 14:58:22 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

401.0 step1

runTheMatrix-results/401.0_TTbarNewMix+TTbarFSPU2+HARVESTFS/step1_TTbarNewMix+TTbarFSPU2+HARVESTFS.log
----- Begin Fatal Exception 28-Jan-2014 14:58:22 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

5.1 step1

runTheMatrix-results/5.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
----- Begin Fatal Exception 28-Jan-2014 14:58:23 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

1003.0 step2

runTheMatrix-results/1003.0_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM/step2_RunMinBias2012A+RunMinBias2012A+RECODDQM+HARVESTDDQM.log
----- Begin Fatal Exception 28-Jan-2014 14:58:26 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODreHLT+HARVESTDreHLT.log
----- Begin Fatal Exception 28-Jan-2014 15:01:58 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log
----- Begin Fatal Exception 28-Jan-2014 15:02:30 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECO+HARVEST+ALCATT.log
----- Begin Fatal Exception 28-Jan-2014 15:09:34 CET-----------------------
An exception of category 'PFRecoTauDiscriminationAgainstElectronMVA5GBR' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=PFRecoTauDiscriminationAgainstElectronMVA5GBR label='hpsPFTauDiscriminationByMVA5rawElectronRejection'
Exception Message:
 Failed to find File = V001 RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root 2 /src/RecoTauTag/RecoTau/data/gbrDiscriminationAgainstElectronMVA5.root !!
----- End Fatal Exception -------------------------------------------------

you can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2190/4/summary.html

@fwyzard
Copy link
Contributor Author

fwyzard commented Jan 28, 2014

Mhm, I guess this is unrelated ?

@ktf
Copy link
Contributor

ktf commented Jan 28, 2014

Yes, will be fixed in next IB. See discussion in HN.

@cmsbuild
Copy link
Contributor

+1
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2190/21/summary.html

@deguio
Copy link
Contributor

deguio commented Feb 5, 2014

+1
I think this should go in 71X though

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 5, 2014

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_0_X IBs unless changes (tests are also fine). @nclopezo, @ktf can you please take care of it?

@davidlange6
Copy link
Contributor

yes - 7_1_X would be best. (though it looks like just a new function - so perhaps its not called by default?)

On Feb 5, 2014, at 4:42 PM, cmsbuild notifications@github.com
wrote:

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_0_X IBs unless changes (tests are also fine). @nclopezo, @ktf can you please take care of it?

—
Reply to this email directly or view it on GitHub.

@deguio
Copy link
Contributor

deguio commented Feb 5, 2014

@davidlange6
it is not called by default. may be we can put it in.. it will not hurt.

@Dr15Jones
Copy link
Contributor

I can confirm that the function is only called in the case where cmsRun has been told to fork, which is not a way we every have run in production.

@fwyzard
Copy link
Contributor Author

fwyzard commented Feb 5, 2014

Hi all,
please include this in 70x, as it's needed for running the multi-process
cmsRun, which is the only way to fully use all cores on the online machines
until multithreading is working.
Also, it is already included (and used) in 62x.

Thanks,
.Andrea
On 5 Feb 2014 16:58, "Chris Jones" notifications@github.com wrote:

I can confirm that the function is only called in the case where cmsRun
has been told to fork, which is not a way we every have run in production.

Reply to this email directly or view it on GitHubhttps://github.com//pull/2190#issuecomment-34195291
.

@davidlange6
Copy link
Contributor

+1

On Feb 5, 2014, at 6:28 PM, Andrea Bocci notifications@github.com
wrote:

Hi all,
please include this in 70x, as it's needed for running the multi-process
cmsRun, which is the only way to fully use all cores on the online machines
until multithreading is working.
Also, it is already included (and used) in 62x.

Thanks,
.Andrea
On 5 Feb 2014 16:58, "Chris Jones" notifications@github.com wrote:

I can confirm that the function is only called in the case where cmsRun
has been told to fork, which is not a way we every have run in production.

Reply to this email directly or view it on GitHubhttps://github.com//pull/2190#issuecomment-34195291
.

—
Reply to this email directly or view it on GitHub.

ktf added a commit that referenced this pull request Feb 6, 2014
@ktf
Merge pull request #2190 from fwyzard/multiProcesses_DQMRootOutputMod…
3f55a47 
…ule_for_70x

DQM -- DQMRootOutputModule: implement support for multi-process cmsRun
@ktf ktf merged commit 3f55a47 into cms-sw:CMSSW_7_0_X Feb 6, 2014
@fwyzard fwyzard deleted the multiProcesses_DQMRootOutputModule_for_70x branch February 13, 2014 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
dqm-approved fully-signed tests-approved
Projects
None yet
Milestone
CMSSW_7_0_0
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@fwyzard @cmsbuild @ktf @deguio @davidlange6 @Dr15Jones @nclopezo