Make mass replace work with proxy types #30164

Dr15Jones · 2020-06-09T20:11:04Z

PR description:

Adding isCompatibleCMSType allows handling of proxy types in mass search functions involving InputTags

PR validation:

The unit tests pass.

Adding isCompatibleCMSType allows handling of proxy types in mass search functions involving InputTags

cmsbuild · 2020-06-09T20:11:27Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-06-09T20:17:05Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30164/15965

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/ParameterSet/python/MassReplace.py modified in PR(s): skip complicated checks for PFCandidate-on-PFCandidate TopProjector #30071
- File FWCore/ParameterSet/python/Mixins.py modified in PR(s): Allow InputTag and ESInputTag in a PSet to be assigned from a sequence of strings #29992
- File FWCore/ParameterSet/python/Types.py modified in PR(s): Allow InputTag and ESInputTag in a PSet to be assigned from a sequence of strings #29992

cmsbuild · 2020-06-09T20:17:24Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

FWCore/ParameterSet

@makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @wddgit this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2020-06-09T20:22:00Z

please test

cmsbuild · 2020-06-09T20:22:21Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6914/console Started: 2020/06/09 22:26

cmsbuild · 2020-06-09T21:54:21Z

-1

Tested at: 3768603

CMSSW: CMSSW_11_2_X_2020-06-09-1100
SCRAM_ARCH: slc7_amd64_gcc820

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

Removed boost::uit64_t , they are now built-in c++ #30048 @camolezi: Removed boost::uit64_t , they are now built-in c++
EMTF: Prevent out-of-bounds access in BDT event data #30051 @jiafulow: EMTF: Prevent out-of-bounds access in BDT event data
added esConsumes to modules in SimMuonCSCDigitizer #30156 @JamminJones: added esConsumes to modules in SimMuonCSCDigitizer
make DetStatus a stream module #30124 @mmusich: make DetStatus a stream module
Run3-alca167 Update the analyzer to study low luminosity collision data #30108 @bsunanda: Run3-alca167 Update the analyzer to study low luminosity collision data
fix direct access to tau ID container #30106 @cms-tau-pog: fix direct access to tau ID container
Phase2-hgx242 First step to use ASCII file due to Philippe and Katja to define HGCal geometry #30079 @bsunanda: Phase2-hgx242 First step to use ASCII file due to Philippe and Katja to define HGCal geometry

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e8128f/6914/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e8128f/6914/git-merge-result

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e8128f/6914/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests RelVals

Unit Tests:

I found errors in the following unit tests:

---> test runtestPhysicsToolsPatAlgos had ERRORS
---> test TestConfigDP had ERRORS

RelVals:

When I ran the RelVals I found an error in the following workflows:
140.53 step2

runTheMatrix-results/140.53_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI/step2_RunHI2011+RunHI2011+RECOHID11+HARVESTDHI.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

Removed boost::uit64_t , they are now built-in c++ #30048 @camolezi: Removed boost::uit64_t , they are now built-in c++
EMTF: Prevent out-of-bounds access in BDT event data #30051 @jiafulow: EMTF: Prevent out-of-bounds access in BDT event data
added esConsumes to modules in SimMuonCSCDigitizer #30156 @JamminJones: added esConsumes to modules in SimMuonCSCDigitizer
make DetStatus a stream module #30124 @mmusich: make DetStatus a stream module
Run3-alca167 Update the analyzer to study low luminosity collision data #30108 @bsunanda: Run3-alca167 Update the analyzer to study low luminosity collision data
fix direct access to tau ID container #30106 @cms-tau-pog: fix direct access to tau ID container
Phase2-hgx242 First step to use ASCII file due to Philippe and Katja to define HGCal geometry #30079 @bsunanda: Phase2-hgx242 First step to use ASCII file due to Philippe and Katja to define HGCal geometry

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e8128f/6914/git-log-recent-commits
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e8128f/6914/git-merge-result

cmsbuild · 2020-06-09T21:54:24Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

slava77 · 2020-06-09T23:19:28Z

FWCore/ParameterSet/python/MassReplace.py

@@ -38,7 +38,7 @@ def doIt(self,pset,base):
                            nrep = n; nrep.moduleLabel = self._paramReplace.moduleLabel
                            if self._verbose:print("Replace %s.%s[%d] %s ==> %s " % (base, name, i, n, nrep))
                            value[i] = nrep
-                elif isinstance(value, cms.InputTag):
+                elif value.isCompatibleCMSType(cms.InputTag):
                    if value == self._paramSearch:


I suspect that something like eqWproxy from https://github.com/cms-sw/cmssw/pull/30071/files#diff-c2356aa3d79bc5ad0d7e89ea86cca856R12-R16 will be needed here, and apparently in the VInputTag case above

this and value is more elegant

cmsbuild · 2020-06-11T23:16:23Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e8128f/6977/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2779433
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2779382
DQMHistoTests: Total skipped: 50
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
Checked 152 log files, 16 edm output root files, 36 DQM output files

makortel · 2020-06-11T23:20:13Z

+1

cmsbuild · 2020-06-11T23:20:51Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2020-06-12T02:15:58Z

The tests are being triggered in jenkins.
Tested with other pull request(s) #30071
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/6981/console Started: 2020/06/12 09:52

silviodonato · 2020-06-12T08:43:55Z

+1

silviodonato · 2020-06-12T08:52:42Z

merge

cmsbuild · 2020-06-12T09:53:43Z

+1
Tested at: 19f57b6
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2bb31f/6981/summary.html
CMSSW: CMSSW_11_2_X_2020-06-11-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-06-12T09:53:47Z

Comparison job queued.

cmsbuild · 2020-06-12T11:46:40Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2bb31f/6981/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 36
DQMHistoTests: Total histograms compared: 2779433
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2779382
DQMHistoTests: Total skipped: 50
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 35 files compared)
Checked 152 log files, 16 edm output root files, 36 DQM output files

Make mass replace work with proxy types

3768603

Adding isCompatibleCMSType allows handling of proxy types in mass search functions involving InputTags

cmsbuild added this to the CMSSW_11_2_X milestone Jun 9, 2020

cmsbuild added code-checks-pending comparison-pending core-pending orp-pending pending-signatures tests-pending labels Jun 9, 2020

Dr15Jones mentioned this pull request Jun 9, 2020

skip complicated checks for PFCandidate-on-PFCandidate TopProjector #30071

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 9, 2020

cmsbuild added tests-started and removed tests-pending labels Jun 9, 2020

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Jun 9, 2020

slava77 reviewed Jun 9, 2020

View reviewed changes

cmsbuild mentioned this pull request Jun 10, 2020

Allow InputTag and ESInputTag in a PSet to be assigned from a sequence of strings #29992

Merged

Handle case were optional parameter is not set

19f57b6

cmsbuild removed code-checks-approved comparison-notrun tests-rejected labels Jun 11, 2020

cmsbuild added tests-approved and removed tests-started labels Jun 11, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jun 11, 2020

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Jun 11, 2020

cmsbuild added requires-external tests-started and removed tests-approved labels Jun 12, 2020

cmsbuild added orp-approved and removed orp-pending labels Jun 12, 2020

cmsbuild merged commit c7ad92e into cms-sw:master Jun 12, 2020

cmsbuild added comparison-pending tests-approved and removed comparison-available tests-started labels Jun 12, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jun 12, 2020

Dr15Jones deleted the betterMassReplace branch June 12, 2020 13:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make mass replace work with proxy types #30164

Make mass replace work with proxy types #30164

Dr15Jones commented Jun 9, 2020

cmsbuild commented Jun 9, 2020

cmsbuild commented Jun 9, 2020

cmsbuild commented Jun 9, 2020

Dr15Jones commented Jun 9, 2020

cmsbuild commented Jun 9, 2020 •

edited

Loading

cmsbuild commented Jun 9, 2020

cmsbuild commented Jun 9, 2020

slava77 Jun 9, 2020

slava77 Jun 12, 2020

cmsbuild commented Jun 11, 2020

makortel commented Jun 11, 2020

cmsbuild commented Jun 11, 2020

cmsbuild commented Jun 12, 2020 •

edited

Loading

silviodonato commented Jun 12, 2020

silviodonato commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

Make mass replace work with proxy types #30164

Make mass replace work with proxy types #30164

Conversation

Dr15Jones commented Jun 9, 2020

PR description:

PR validation:

cmsbuild commented Jun 9, 2020

cmsbuild commented Jun 9, 2020

cmsbuild commented Jun 9, 2020

Dr15Jones commented Jun 9, 2020

cmsbuild commented Jun 9, 2020 • edited Loading

cmsbuild commented Jun 9, 2020

cmsbuild commented Jun 9, 2020

slava77 Jun 9, 2020

Choose a reason for hiding this comment

slava77 Jun 12, 2020

Choose a reason for hiding this comment

cmsbuild commented Jun 11, 2020

makortel commented Jun 11, 2020

cmsbuild commented Jun 11, 2020

cmsbuild commented Jun 12, 2020 • edited Loading

silviodonato commented Jun 12, 2020

silviodonato commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

cmsbuild commented Jun 12, 2020

cmsbuild commented Jun 9, 2020 •

edited

Loading

cmsbuild commented Jun 12, 2020 •

edited

Loading