[GENERATORS] Update all the scripts to use python3

[smuzaffar]

Cleanup python2 usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-37105/28600

• This PR adds an extra 24KB to repository

[cmsbuild]

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for master.

It involves the following packages:

• GeneratorInterface/Herwig7Interface (generators)
• GeneratorInterface/LHEInterface (generators)

@SiewYan, @mkirsano, @Saptaparna, @cmsbuild, @GurpreetSinghChahal, @alberto-sanchez can you please review it and eventually sign? Thanks.
@alberto-sanchez, @mkirsano this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

[Saptaparna]

@Dominic-Stafford: does this affect anything significantly regarding Herwig?
@SiewYan: Do we need to implement further checks for python 3?

Thanks!

[SiewYan]

@Saptaparna I don't think so for now, it is included in the unit test.

[SiewYan]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-479fd8/22755/summary.html
COMMIT: 66154ad
CMSSW: CMSSW_12_3_X_2022-03-01-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37105/22755/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 3 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 4000857
• DQMHistoTests: Total failures: 8
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 4000827
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[SiewYan]

@Saptaparna , there is no python error/warning found in the tests [*],[**].

[*] https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-479fd8/22755/clang-logs/
[**] https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-479fd8/22755/build-logs/

[smuzaffar]

@SiewYan , please check the unit tests e.g.

• cmsLHEtoEOSManager.py: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-479fd8/22755/unitTests/src/GeneratorInterface/LHEInterface/test/test_cmsLHEtoEOSManager/testing.log
• https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-479fd8/22755/unitTests/src/GeneratorInterface/LHEInterface/test/testGeneratorInterfaceLHEInterface_TP/testing.log

is there any unit test for testing GeneratorInterface/LHEInterface/scripts/mergeLHE.py and GeneratorInterface/Herwig7Interface/scripts/parallelization.py

[smuzaffar]

by the way, do we know why https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-479fd8/22755/unitTests/src/GeneratorInterface/LHEInterface/test/testGeneratorInterfaceLHEInterface_TP/testing.log has some exceptions . These are already there in the IBs too. Are these expected and if not then why unit test is not failing and exiting with non-zero exit code?

[Dominic-Stafford]

@Dominic-Stafford: does this affect anything significantly regarding Herwig?

No, the only change on the Herwig side is to the parallelisation.py script, which isn't currently used for anything as far as I know.

[SiewYan]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-479fd8/22794/summary.html
COMMIT: 66154ad
CMSSW: CMSSW_12_3_X_2022-03-02-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/37105/22794/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3987741
• DQMHistoTests: Total failures: 14
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3987705
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 204 log files, 45 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[SiewYan]

@smuzaffar thanks for catching this error, apparently there is a bug on the unitest code, the file format is different, cmsLHEtoEOSManager.py.

I will address this later this week.

[smuzaffar]

@cms-sw/generators-l2 can you please review this? I think this looks good and does not break any thing new :-)

[SiewYan]

@smuzaffar , sure no problem! i will work on this tiny issue from another PR. Thanks for pointing out though.

[SiewYan]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1