Python3: changes needed to fix runTheMatrix errors for PY3 IBs

[smuzaffar]

This change should allow to run most of relvals for Python3 IBs. There are still some errors but those are coming from external packages.

[cmsbuild]

The code-checks are being triggered in jenkins.

[smuzaffar]

please test workflow 200.0,1370.0 for CMSSW_11_0_PY3_X

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28078/12039

• This PR adds an extra 40KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Applications/python/ConfigBuilder.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2695/console Started: 2019/09/27 09:27

[cmsbuild]

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for master.

It involves the following packages:

Configuration/Applications
GeneratorInterface/LHEInterface

@SiewYan, @efeyazgan, @mkirsano, @kpedro88, @cmsbuild, @franzoni, @agrohsje, @alberto-sanchez, @qliphy, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@makortel, @alberto-sanchez, @agrohsje, @mkirsano, @Martin-Grunewald this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

[cmsbuild]

The code-checks are being triggered in jenkins.

[smuzaffar]

please test workflow 200.0,1370.0 for CMSSW_11_0_PY3_X

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28078/12043

• This PR adds an extra 44KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/Applications/python/ConfigBuilder.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2699/console Started: 2019/09/27 09:52

[cmsbuild]

+1
Tested at: 9487130
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d4a3c6/2702/summary.html

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d4a3c6/2702/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2958092
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2957750
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
• Checked 145 log files, 15 edm output root files, 34 DQM output files

[smuzaffar]

+core

[fabiocos]

+operations

@smuzaffar the addition of universal_newlines allows the correct reading of text files, am I correct?

[smuzaffar]

Yes, universal_newlines is to get the data in text format. By default it is in binary format.

protocol=2 option is needed to generate pickle files in format which python2 can also read it. addOnTest generates a pickle file with summary of results which we upload to cmssdt web server where we have python2.

[smuzaffar]

@SiewYan, @efeyazgan @qliphy can you please look in to it? This is needed to fix many failing workflows for PY3 IBs.

[qliphy]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

[smuzaffar]

@fabiocos , @kpedro88 can we please get this in IBs?

[kpedro88]

+1