Python2to3: cmp to key conversion in list sort substitute #28080

mrodozov · 2019-09-27T13:41:58Z

PR description:

Fixes python3 implementation of list sorting using a comparison function
as described here:
https://stackoverflow.com/questions/2531952/how-to-use-a-custom-comparison-function-in-python-3

PR validation:

Works both with py2 and py3

mrodozov · 2019-09-27T13:42:12Z

please test

cmsbuild · 2019-09-27T13:42:30Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-09-27T13:49:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28080/12047

This PR adds an extra 20KB to repository

cmsbuild · 2019-09-27T13:50:25Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2703/console Started: 2019/09/27 15:51

cmsbuild · 2019-09-27T13:50:26Z

A new Pull Request was created by @mrodozov (Mircho Rodozov) for master.

It involves the following packages:

FWCore/Concurrency

@cmsbuild, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @wddgit this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2019-09-27T15:23:58Z

+1
Tested at: a55820c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-238fb3/2703/summary.html

cmsbuild · 2019-09-27T15:24:02Z

Comparison job queued.

cmsbuild · 2019-09-27T17:33:51Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-238fb3/2703/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2958092
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2957750
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 145 log files, 15 edm output root files, 34 DQM output files

Dr15Jones · 2019-10-02T13:38:32Z

How was this tested ?

mrodozov · 2019-10-02T14:15:10Z

This test
TestFWCoreConcurrencyDriver
was failing in
cms-sw/cmsdist#5241
and using the PY3 IB I tried this suggestion, tried also with py2 IB and also worked.

smuzaffar · 2019-10-02T14:16:35Z

FWCore/Concurrency/scripts/edmStreamStallGrapher.py

@@ -424,7 +424,7 @@ def printStalledModulesInOrder(stalledModules):

    def sumSort(i,j):
        return cmp(i[1],j[1])
-    priorities.sort(cmp=sumSort, reverse=True)
+    priorities.sort(key=cmp_to_key(sumSort), reverse=True)


following should work for both py2 and 3

priorities = sorted(priorities , key=lambda a: a[1], reverse=True)

FWCore/Concurrency/scripts/edmStreamStallGrapher.py

commit suggestion as proposed on github (and test commit suggestion feature ) Co-Authored-By: Chris Jones <chrisdjones15@gmail.com>

cmsbuild · 2019-10-14T12:03:47Z

The code-checks are being triggered in jenkins.

cmsbuild · 2019-10-14T12:10:09Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28080/12232

This PR adds an extra 24KB to repository

cmsbuild · 2019-10-14T12:10:33Z

Pull request #28080 was updated. @cmsbuild, @smuzaffar, @Dr15Jones can you please check and sign again.

mrodozov · 2019-10-14T12:17:28Z

please test

cmsbuild · 2019-10-14T12:17:53Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2915/console Started: 2019/10/14 14:18

Dr15Jones · 2019-10-14T13:29:09Z

+1

cmsbuild · 2019-10-14T13:29:36Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2019-10-14T14:49:38Z

+1
Tested at: 72ca248
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-238fb3/2915/summary.html

cmsbuild · 2019-10-14T14:49:42Z

Comparison job queued.

cmsbuild · 2019-10-14T17:12:31Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-238fb3/2915/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2961064
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2960722
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

fabiocos · 2019-10-14T20:46:28Z

+1

Python2to3: cmp to key conversion in list sort substitute

a55820c

cmsbuild added this to the CMSSW_11_0_X milestone Sep 27, 2019

cmsbuild added code-checks-pending comparison-pending core-pending orp-pending pending-signatures tests-pending labels Sep 27, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 27, 2019

cmsbuild added tests-started and removed tests-pending labels Sep 27, 2019

cmsbuild added tests-approved and removed tests-started labels Sep 27, 2019

cmsbuild added comparison-available and removed comparison-pending labels Sep 27, 2019

smuzaffar reviewed Oct 2, 2019

View reviewed changes

Dr15Jones reviewed Oct 8, 2019

View reviewed changes

FWCore/Concurrency/scripts/edmStreamStallGrapher.py Outdated Show resolved Hide resolved

Update FWCore/Concurrency/scripts/edmStreamStallGrapher.py

88e6238

commit suggestion as proposed on github (and test commit suggestion feature ) Co-Authored-By: Chris Jones <chrisdjones15@gmail.com>

cmsbuild removed the code-checks-approved label Oct 14, 2019

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-started labels Oct 14, 2019

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 14, 2019

cmsbuild added tests-started and removed tests-pending labels Oct 14, 2019

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Oct 14, 2019

cmsbuild added tests-approved and removed tests-started labels Oct 14, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 14, 2019

cmsbuild added orp-approved and removed orp-pending labels Oct 14, 2019

cmsbuild merged commit a60d596 into cms-sw:master Oct 14, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python2to3: cmp to key conversion in list sort substitute #28080

Python2to3: cmp to key conversion in list sort substitute #28080

mrodozov commented Sep 27, 2019

mrodozov commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019 •

edited

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

Dr15Jones commented Oct 2, 2019

mrodozov commented Oct 2, 2019

smuzaffar Oct 2, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

mrodozov commented Oct 14, 2019

cmsbuild commented Oct 14, 2019 •

edited

Dr15Jones commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

fabiocos commented Oct 14, 2019

Python2to3: cmp to key conversion in list sort substitute #28080

Python2to3: cmp to key conversion in list sort substitute #28080

Conversation

mrodozov commented Sep 27, 2019

PR description:

PR validation:

mrodozov commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019 • edited

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

cmsbuild commented Sep 27, 2019

Dr15Jones commented Oct 2, 2019

mrodozov commented Oct 2, 2019

smuzaffar Oct 2, 2019

Choose a reason for hiding this comment

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

mrodozov commented Oct 14, 2019

cmsbuild commented Oct 14, 2019 • edited

Dr15Jones commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

cmsbuild commented Oct 14, 2019

fabiocos commented Oct 14, 2019

cmsbuild commented Sep 27, 2019 •

edited

cmsbuild commented Oct 14, 2019 •

edited