Updates pip based packages

[smuzaffar]

• new versions for pip based python packages
• Use --upgrade --upgrade-strategy only-if-needed to make sure that we always install a package in our externals area.
• Cleanup for py2 and py3 packages
• Cleanup of some unused files
• Create lib/py3 for llvm
• Build xrootd for both python2 and python3
• Build numpy, matplotlib and tensorflow using our build-with-pip system.
• Build dxr for py3
• Get cx-Oracle from pip and install it for both py2 and py3

[smuzaffar]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2754/console Started: 2019/10/02 15:49

[cmsbuild]

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_11_0_X/master.

@cmsbuild, @smuzaffar, @mrodozov can you please review it and eventually sign? Thanks.
cms-bot commands are listed here

[smuzaffar]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2756/console Started: 2019/10/02 15:58

[cmsbuild]

Pull request #5256 was updated.

[cmsbuild]

-1

Tested at: cc974fe

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e81285/2756/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

• Unit Tests:

I found errors in the following unit tests:

---> test testTheano had ERRORS
---> test testImports had ERRORS

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e81285/2756/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2956833
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2956491
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
• Checked 147 log files, 16 edm output root files, 34 DQM output files

[smuzaffar]

please test

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e81285/2780/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 8 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2961049
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2960706
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
• Checked 147 log files, 16 edm output root files, 34 DQM output files

[cmsbuild]

-1

Tested at: f9dd471

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e81285/2796/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

• Unit Tests:

I found errors in the following unit tests:

---> test test_MpsWorkFlow had ERRORS
---> test testPyDevParameterSet had ERRORS
---> test testTauEmbeddingProducers had ERRORS
---> test ExpressionEvaluatorUnitTest had ERRORS

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e81285/2796/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 14 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2961049
• DQMHistoTests: Total failures: 129
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2960579
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
• Checked 147 log files, 16 edm output root files, 34 DQM output files

[cmsbuild]

-1

Tested at: f9dd471

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e81285/2801/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests

• Unit Tests:

I found errors in the following unit tests:

---> test test_MpsWorkFlow had ERRORS
---> test ExpressionEvaluatorUnitTest had ERRORS

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e81285/2801/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 2 differences found in the comparisons
• DQMHistoTests: Total files compared: 34
• DQMHistoTests: Total histograms compared: 2961049
• DQMHistoTests: Total failures: 2
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2960706
• DQMHistoTests: Total skipped: 341
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
• Checked 147 log files, 16 edm output root files, 34 DQM output files

[smuzaffar]

+externals
failing unit tests has nothing to do with this change. Unit tests itself need fixing.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_11_0_X/master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)