Python package updates / addition of lz4

[davidlange6]

adding lz4 instead of picking up from inside of root
adding python interfaces to lz4 and lzma
updating to latest version of a number of python packages

[cmsbuild]

A new Pull Request was created by @davidlange6 (David Lange) for branch IB/CMSSW_10_2_X/gcc630.

@cmsbuild, @smuzaffar, @gudrutis, @mrodozov can you please review it and eventually sign? Thanks.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

[davidlange6]

please test On 9 Jun 2018, at 11:30, David Lange <notifications@github.com<mailto:notifications@github.com>> wrote: adding lz4 instead of picking up from inside of root adding python interfaces to lz4 and lzma updating to latest version of a number of python packages

…

________________________________ You can view, comment on, or merge this pull request online at: #4103 Commit Summary * python version updates, add lzma interface * add lz4 * add lz4 * add lzma python interface * fix spec File Changes * A lz4-toolfile.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-0> (22) * A lz4.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-1> (19) * M py2-Theano.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-2> (2) * A py2-backports_lzma.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-3> (12) * M py2-bleach.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-4> (2) * M py2-certifi.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-5> (2) * M py2-cython.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-6> (2) * M py2-flawfinder.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-7> (2) * M py2-fs.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-8> (2) * M py2-h5py.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-9> (2) * M py2-jupyter_client.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-10> (2) * M py2-llvmlite.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-11> (2) * A py2-lz4.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-12> (11) * M py2-oamap.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-13> (2) * M py2-packaging.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-14> (2) * M py2-pandas.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-15> (2) * M py2-pexpect.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-16> (2) * M py2-psutil.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-17> (2) * M py2-pyopenssl.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-18> (2) * M py2-python-dateutil.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-19> (2) * M py2-pytz.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-20> (2) * M py2-scikit-learn.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-21> (2) * M py2-tqdm.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-22> (2) * M py2-uproot.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-23> (2) * M py2-xgboost.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-24> (16) * M python_tools.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-25> (1) * M root.spec<https://github.com/cms-sw/cmsdist/pull/4103/files#diff-26> (6) Patch Links: * https://github.com/cms-sw/cmsdist/pull/4103.patch * https://github.com/cms-sw/cmsdist/pull/4103.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#4103>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AEzyw8F7cpxM87gIThyM_ApFJeJRAB20ks5t65V4gaJpZM4UhQZu>.

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28597/console

[smuzaffar]

@davidlange6 , why tool name is xz and not lz4?

[cmsbuild]

Pull request #4103 was updated.

[davidlange6]

hi @smuzaffar - thanks for noticing - its a mistake

[davidlange6]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28606/console
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28611/console

[cmsbuild]

Pull request #4103 was updated.

[smuzaffar]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28633/console

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4103/28633/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 31
• DQMHistoTests: Total histograms compared: 2901984
• DQMHistoTests: Total failures: 24
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2901770
• DQMHistoTests: Total skipped: 190
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
• Checked 128 log files, 14 edm output root files, 31 DQM output files

[smuzaffar]

@davidlange6 , can you please check the comparison results?

[smuzaffar]

please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28656/console
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/28658/console

[cmsbuild]

+1
Tested at: d583a2c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4103/28658/summary.html
Additional comment: 'Compilation

[cmsbuild]

Comparison job queued.

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4103/28658/summary.html

Comparison Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 31
• DQMHistoTests: Total histograms compared: 2902004
• DQMHistoTests: Total failures: 25
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 2901789
• DQMHistoTests: Total skipped: 190
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 30 files compared)
• Checked 128 log files, 14 edm output root files, 31 DQM output files

[smuzaffar]

@davidlange6 , can you please check the comparison for lz4 again. It again shows 25 errors.

[davidlange6]

I notice the same sort of failures in #23545 - so I'd conclude its a new code bug in cmssw rather than this PR.

[smuzaffar]

+externals
ok, so I am merging this PR now.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_10_2_X/gcc630 IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)