Fix the mergeLHE.py script

[colizz]

PR description:

Fix the regex to match the LHE tags in the mergeLHE.py script.

Now we are capable to match the tag if it has leading spaces. The issue is discussed in #34531 (comment)

PR validation:

Validation done based on https://github.com/colizz/mergelhe_validate indicated by mergeLHE.py -h .

A backport to 10_6_X is also needed.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-34804/24527

• This PR adds an extra 16KB to repository

[cmsbuild]

A new Pull Request was created by @colizz (Congqiao Li) for master.

It involves the following packages:

• GeneratorInterface/LHEInterface (generators)

@SiewYan, @mkirsano, @Saptaparna, @cmsbuild, @GurpreetSinghChahal, @agrohsje, @alberto-sanchez can you please review it and eventually sign? Thanks.
@alberto-sanchez, @mkirsano this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy, @perrotta you are the release manager for this.

cms-bot commands are listed here

[perrotta]

please test

[smuzaffar]

test parameters:

• workflow_threading = 537.0
• enable_test = threading

[smuzaffar]

please test
re-run with threading and workflow 537.0

[colizz]

Thanks. My local test shows that 537.0 in the 4-thread test will still report an error in mergeLHE.py, because the LHE produced from thread0 is already problematic. The script will find inconsistencies for the LHE headers.

AssertionError: Incompatibility found in LHE headers: line number not matches. Use -b/--bypass-check to bypass the check.

The issue will be solved by changing to event number from 10 to 12, in #34710

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3e4be7/17603/summary.html
COMMIT: 89b78ab
CMSSW: CMSSW_12_1_X_2021-08-05-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34804/17603/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

• 11630.011630.0_QCD_Pt_3000_3500_14TeV+2021+QCD_Pt_3000_3500_14TeV_TuneCUETP8M1_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_QCD_Pt_3000_3500_14TeV+2021+QCD_Pt_3000_3500_14TeV_TuneCUETP8M1_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
• 11601.011601.0_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt10+2021+SingleElectronPt10_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log
• 11603.011603.0_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA/step2_SingleElectronPt1000+2021+SingleElectronPt1000_pythia8_GenSimINPUT+Digi+Reco+HARVEST+ALCA.log

Expand to see more relval errors ...

• 11602.0
• 11604.0
• 11605.0
• 11606.0
• 11607.0
• 11608.0
• 11609.0
• 11634.0
• 11634.1
• 11634.24
• 11634.5
• 11634.501
• 11634.505
• 11634.511
• 11634.521
• 11634.91
• 11634.7
• 11640.0
• 11643.0
• 11646.0
• 11650.0
• 11650.501
• 11650.505
• 11723.17
• 11725.0
• 11834.0
• 11834.13
• 11834.24
• 11846.0
• 11925.0
• 12034.0
• 12434.0
• 12634.0
• 12834.0
• 13034.0
• 23434.99
• 23434.21
• 23434.9921
• 34834.21
• 34834.9921

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2999410
• DQMHistoTests: Total failures: 7
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2999381
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: no differences found

[qliphy]

please test with #34710

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3e4be7/17634/summary.html
COMMIT: 89b78ab
CMSSW: CMSSW_12_1_X_2021-08-08-0000/slc7_amd64_gcc900
Additional Tests: THREADING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/34804/17634/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 3 differences found in the comparisons
• DQMHistoTests: Total files compared: 39
• DQMHistoTests: Total histograms compared: 2999410
• DQMHistoTests: Total failures: 7
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2999381
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 38 files compared)
• Checked 165 log files, 37 edm output root files, 39 DQM output files
• TriggerResults: no differences found

[qliphy]

@cms-sw/generators-l2 Any comment?

[agrohsje]

+generators
Thanks for the prompt fix @colizz .

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1

[perrotta]

fixes #34531

[perrotta]

@colizz @agrohsje : if this fix needs to be backported, please prepare a PR for 10_6 (as you write in the PR description) and for 12_0_X (where workflow 537 is still failing)

[colizz]

Thanks @perrotta, the failure of workflow 537 will only be fixed by https://github.com/cms-sw/cmssw/pull/34710/files/ce06076dc65aadc045eb878c99ee0ef3f88cd48a..2e7e1c0ca67865630b9aae21948cf75b6ffad0e5
by changing the event number from 10 to 12. (PS. the error is caused by a physics issue in the generator meaning that generating merely 2 events may probabilistically cause zero x-section)

This PR only helps to make the script more generalized. But I'll backport both of them to help 537 works.

[colizz]

Please see 12_0_X backport in #34861.
The 10_6_X backports are covered in #34527 (specifically, in bullet-7 and 8)