Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding first LHE sequence to RelVal #6626

Merged
merged 14 commits into from Nov 28, 2014
Merged

Adding first LHE sequence to RelVal #6626

merged 14 commits into from Nov 28, 2014

Conversation

cmsbuild
Copy link
Contributor

This pull request include the first RelVal workflows with LHE for DY->ll+Jets. Moreover, following the request of Josh, the jet matching has been changed to use the new pythia8 jet matching for the LHE workflows. All of RelVal sequences that use LHE files in the generator and extended generator workflows have had the a fragment added for easy use in the future. The full generator and extended RelVal workflows have been tested on a small number of events.
Automatically ported from CMSSW_7_3_X #6554

inugent added 11 commits November 17, 2014 16:32
…adgraph_LHE_13TeV_cff.py as first step for LHE generation in RelVal. 11/17/2014 (nugent@physik.rwth-aachen.de)
…f.py to convert lhe to hep format. (first version - issue with missing LHERunInfoProduct)
…in relval generator sequence... + added to generator sequence
…or this pull request + patches to lhe relval found during testing(validation in progress - may have more).
…o new files and adding originals back. Adding patch to new Hadronizer_MgmMatchTuneCUETP8M1JetMatching*. Modifying relval_steps.py for this update.
…the DQM file when running 2 spteps before harvesting. <nugent@physik.rwth-aachen.de>
…uneCUETP8M1_8TeV_MLM_5f_max4j_LHE_pythia8* and fixing nJetMax=4 new gen frag. This update also include changes to the naming to the relval sequences to match this new convention. <nugent@physik.rwth-aachen.de>
…TeV in file names and in workflow). <nugent@physik.rwth-aachen.de>
…TeV in file names and in workflow). <nugent@physik.rwth-aachen.de>
…ll0123->DYToll01234 in relval_steps.py. <nugent@physik.rwth-aachen.de>
@cmsbuild
Copy link
Contributor Author

A new Pull Request was created by @cmsbuild for CMSSW_7_4_X.

Adding first LHE sequence to RelVal

It involves the following packages:

Configuration/Generator
Configuration/PyReleaseValidation

@vciulli, @covarell, @boudoul, @franzoni, @thuer, @cmsbuild, @srimanob, @nclopezo, @bendavid can you please review it and eventually sign? Thanks.
@ghellwig this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

@nclopezo
Copy link
Contributor

+1
Latest commit marked as "Tests OK" in CMSSW_7_3_X

…run1_mc to auto:run2_mc_FULL; 2) switch to auto:run2_mc_Full also HARVESTGEN; 3) change HARVESTGEN2 to steps['HARVESTGEN2']=merge([{'--filein':'file:step2_inDQM.root'},steps['HARVESTGEN']]) + setting q cut to 20GeV at Josh's request
@franzoni
Copy link

+1
for local running and running in IB this PR is good
The required changes for the relval production will be placed in a separate PR, coming in minutes. I'll reference here too.

@cmsbuild
Copy link
Contributor Author

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs unless changes (tests are also fine). This pull request will be automatically merged.

@davidlange6
Copy link
Contributor

Seems this brings IB failures in GEN workflows (for reasons not clear to me). Could you investigate? Thanks

@bendavid

@bendavid
Copy link
Contributor

bendavid commented Dec 3, 2014

Hi David,
Can I see the errors somewhere? This workflow (at least step1) requires CVMFS, is this the likely cause/do we need to revisit this?

@bendavid
Copy link
Contributor

bendavid commented Dec 4, 2014

Hi David,
It works for me interactively (on cmsdev07).

Indeed the log is really strange because there really isn't any apparent error in the madgraph step, the ls command at the end shows that the final lhe file IS in fact present.

The next output is supposed to be the event loop, but cmsRun just stops for some reason!?

Which machines do these tests run on? Is it possible for me to run there interactively?

pdf reweighted cross-section :96 17206.4
pdf reweighted cross-section :97 19728
pdf reweighted cross-section :98 17454.8
pdf reweighted cross-section :99 18382.4
pdf reweighted cross-section :100 18026.9
min/max:16161.1 148060
total 164
-rw-r--r--. 1 bendavid zh 104322 Dec 4 01:57 cmsgrid_final.lhe
-rw-r--r--. 1 bendavid zh 40612 Dec 4 01:57 events_presys.lhe
drwxr-xr-x. 15 bendavid zh 4096 Oct 11 11:52 mgbasedir
drwxr-xr-x. 3 bendavid zh 4096 Dec 4 01:57 process
-rwxr-xr-x. 1 bendavid zh 2134 Oct 12 01:23 runcmsgrid.sh
-rw-r--r--. 1 bendavid zh 383 Dec 4 01:57 syscalc_card.dat

Begin processing the 1st record. Run 1, Event 1, LumiSection 1 at 04-Dec-2014 01:57:58.047 CET
%MSG-w MemoryCheck: source 04-Dec-2014 01:57:58 CET PostProcessEvent
MemoryCheck: module source:source VSIZE 617.719 0 RSS 286.578 0.0117188

%MSG
Begin processing the 2nd record. Run 1, Event 2, LumiSection 1 at 04-Dec-2014 01:57:58.060 CET
%MSG-w MemoryCheck: ExternalLHEProducer:externalLHEProducer 04-Dec-2014 01:57:58 CET Run: 1 Event: 2
MemoryCheck: module ExternalLHEProducer:externalLHEProducer VSIZE 617.719 0 RSS 286.59 0.0117188

%MSG
%MSG-w MemoryCheck: PoolOutputModule:LHEoutput 04-Dec-2014 01:57:58 CET Run: 1 Event: 2
MemoryCheck: module PoolOutputModule:LHEoutput VSIZE 617.719 0 RSS 286.594 0.00390625

%MSG
Begin processing the 3rd record. Run 1, Event 3, LumiSection 1 at 04-Dec-2014 01:57:58.064 CET
%MSG-w MemoryCheck: PoolOutputModule:LHEoutput 04-Dec-2014 01:57:58 CET Run: 1 Event: 3
MemoryCheck: module PoolOutputModule:LHEoutput VSIZE 617.719 0 RSS 286.598 0.00390625

%MSG
Begin processing the 4th record. Run 1, Event 4, LumiSection 1 at 04-Dec-2014 01:57:58.068 CET
Begin processing the 5th record. Run 1, Event 5, LumiSection 1 at 04-Dec-2014 01:57:58.071 CET
%MSG-w MemoryCheck: PoolOutputModule:LHEoutput 04-Dec-2014 01:57:58 CET Run: 1 Event: 5
MemoryCheck: module PoolOutputModule:LHEoutput VSIZE 617.969 0.25 RSS 286.762 0.164062

%MSG
Begin processing the 6th record. Run 1, Event 6, LumiSection 1 at 04-Dec-2014 01:57:58.076 CET
%MSG-w MemoryCheck: ExternalLHEProducer:externalLHEProducer 04-Dec-2014 01:57:58 CET Run: 1 Event: 6
MemoryCheck: module ExternalLHEProducer:externalLHEProducer VSIZE 617.969 0 RSS 286.809 0.046875

%MSG
Begin processing the 7th record. Run 1, Event 7, LumiSection 1 at 04-Dec-2014 01:57:58.079 CET
Begin processing the 8th record. Run 1, Event 8, LumiSection 1 at 04-Dec-2014 01:57:58.082 CET
Begin processing the 9th record. Run 1, Event 9, LumiSection 1 at 04-Dec-2014 01:57:58.086 CET
Begin processing the 10th record. Run 1, Event 10, LumiSection 1 at 04-Dec-2014 01:57:58.089 CET

TrigReport ---------- Event Summary ------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Path Summary ------------
TrigReport Trig Bit# Run Passed Failed Error Name
TrigReport 1 0 10 10 0 0 lhe_step

TrigReport -------End-Path Summary ------------
TrigReport Trig Bit# Run Passed Failed Error Name
TrigReport 0 0 10 10 0 0 endjob_step
TrigReport 0 1 10 10 0 0 LHEoutput_step

TrigReport ---------- Modules in Path: lhe_step ------------
TrigReport Trig Bit# Visited Passed Failed Error Name
TrigReport 1 0 10 10 0 0 externalLHEProducer

TrigReport ------ Modules in End-Path: endjob_step ------------
TrigReport Trig Bit# Visited Passed Failed Error Name
TrigReport 0 0 10 10 0 0 MEtoEDMConverter

TrigReport ------ Modules in End-Path: LHEoutput_step ------------
TrigReport Trig Bit# Visited Passed Failed Error Name
TrigReport 0 0 10 10 0 0 LHEoutput

TrigReport ---------- Module Summary ------------
TrigReport Visited Run Passed Failed Error Name
TrigReport 10 10 10 0 0 externalLHEProducer
TrigReport 10 10 10 0 0 TriggerResults
TrigReport 10 10 10 0 0 MEtoEDMConverter
TrigReport 10 10 10 0 0 LHEoutput

TimeReport ---------- Event Summary ---[sec]----
TimeReport CPU/event = 0.003688 Real/event = 0.004515

TimeReport ---------- Path Summary ---[sec]----
TimeReport per event per path-run
TimeReport CPU Real CPU Real Name
TimeReport 0.002736 0.002743 0.002736 0.002743 lhe_step
TimeReport CPU Real CPU Real Name
TimeReport per event per path-run

TimeReport -------End-Path Summary ---[sec]----
TimeReport per event per endpath-run
TimeReport CPU Real CPU Real Name
TimeReport 0.000026 0.000028 0.000026 0.000028 endjob_step
TimeReport 0.000672 0.001494 0.000672 0.001494 LHEoutput_step
TimeReport CPU Real CPU Real Name
TimeReport per event per endpath-run

TimeReport ---------- Modules in Path: lhe_step ---[sec]----
TimeReport per event per module-visit
TimeReport CPU Real CPU Real Name
TimeReport 0.002654 0.002661 0.002654 0.002661 externalLHEProducer
TimeReport CPU Real CPU Real Name
TimeReport per event per module-visit

TimeReport ------ Modules in End-Path: endjob_step ---[sec]----
TimeReport per event per module-visit
TimeReport CPU Real CPU Real Name
TimeReport 0.000001 0.000001 0.000001 0.000001 MEtoEDMConverter

TimeReport ------ Modules in End-Path: LHEoutput_step ---[sec]----
TimeReport per event per module-visit
TimeReport CPU Real CPU Real Name
TimeReport 0.000590 0.001412 0.000590 0.001412 LHEoutput
TimeReport CPU Real CPU Real Name
TimeReport per event per module-visit

TimeReport ---------- Module Summary ---[sec]----
TimeReport per event per module-run per module-visit
TimeReport CPU Real CPU Real CPU Real Name
TimeReport 0.000025 0.000026 0.000025 0.000026 0.000025 0.000026 TriggerResults
TimeReport 0.002654 0.002661 0.002654 0.002661 0.002654 0.002661 externalLHEProducer
TimeReport 0.000001 0.000001 0.000001 0.000001 0.000001 0.000001 MEtoEDMConverter
TimeReport 0.000590 0.001412 0.000590 0.001412 0.000590 0.001412 LHEoutput
TimeReport CPU Real CPU Real CPU Real Name
TimeReport per event per module-run per module-visit

T---Report end!

MemoryReport> Peak virtual size 617.969 Mbytes
Key events increasing vsize:
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[1] run: 1 lumi: 1 event: 1 vsize = 617.719 deltaVsize = 0 rss = 286.57 delta = 0
[5] run: 1 lumi: 1 event: 5 vsize = 617.969 deltaVsize = 0.25 rss = 286.762 delta = 0.191406
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[7] run: 1 lumi: 1 event: 7 vsize = 617.969 deltaVsize = 0 rss = 286.809 delta = 0.046875
[6] run: 1 lumi: 1 event: 6 vsize = 617.969 deltaVsize = 0 rss = 286.809 delta = 0.046875
[5] run: 1 lumi: 1 event: 5 vsize = 617.969 deltaVsize = 0.25 rss = 286.762 delta = 0.191406
TimeReport> Time report complete in 681.928 seconds
Time Summary:

  • Min event: 0.00296998
  • Max event: 0.012167
  • Avg event: 68.1928
  • Total job: 681.928
    CPU Summary:
  • Min event: 0.002999
  • Max event: 0.004
  • Avg event: 0.0035994
  • Total job: 0.961853
  • Total event: 0.035994

MessageLogger Summary

type category sev module subroutine count total


1 MemoryCheck          -w ExternalLHEProdu                       2        2
2 MemoryCheck          -w PoolOutputModule                       3        3
3 MemoryCheck          -w source                                 1        1
4 TimeReport           -e AfterModEndJob                         1        1
5 MemoryReport         -s AfterModEndJob                         1        1

type category Examples: run/evt run/evt run/evt


1 MemoryCheck          1/2              1/6              
2 MemoryCheck          1/2              1/3              1/5
3 MemoryCheck          PostProcessEvent                  
4 TimeReport           PostEndRun                        
5 MemoryReport         PostEndRun                        

Severity # Occurrences Total Occurrences


Warning 6 6
Error 1 1
System 1 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants