drop type specs in RecoTracker/{DeDx,FinalTrackSelectors} #30556

jeongeun · 2020-07-06T15:32:08Z

PR description: Update the safer syntax for existing parameter

Instead of modifying parameters with full type specs, which can be interpreted as an insertion of a new parameter, it is a safer way to protect from parameter name mistakes and will also help in possible parameter migrations.
(The references were PR#29979, PR#30147, PR#30270 , PR#30271, PR#30353)

In this PR, 9 files updated.

RecoTracker/{DeDx} 4 files
RecoTracker/{FinalTrackSelectors} 5 files

PR validation:

Event Content comparison check was also done and there is no change with these updates.
Tested in CMSSW_11_2_X, the basic test all passed in the CMSSW PR instructions.

cmsbuild · 2020-07-06T15:32:29Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-07-06T15:38:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-30556/16792

This PR adds an extra 20KB to repository

cmsbuild · 2020-07-06T15:39:04Z

A new Pull Request was created by @jeongeun (JeongEun Lee) for master.

It involves the following packages:

RecoTracker/DeDx
RecoTracker/FinalTrackSelectors

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @mschrode, @gpetruc, @ebrondol, @mtosi, @dgulhan this is something you requested to watch as well.
@silviodonato, @dpiparo you are the release manager for this.

cms-bot commands are listed here

perrotta · 2020-07-06T15:41:37Z

please test

cmsbuild · 2020-07-06T15:41:56Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-07-06-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/7714/console Started: 2020/07/06 17:46

cmsbuild · 2020-07-06T17:12:03Z

+1
Tested at: 6bb406b
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-eeedea/7714/summary.html
CMSSW: CMSSW_11_2_X_2020-07-06-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-07-06T17:12:06Z

Comparison job queued.

cmsbuild · 2020-07-06T19:01:09Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-eeedea/7714/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2787364
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2787308
DQMHistoTests: Total skipped: 50
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 36 files compared)
DQMHistoSizes: changed ( 10224.0 ): -0.004 KiB MessageLogger/Warnings
Checked 154 log files, 17 edm output root files, 37 DQM output files

perrotta · 2020-07-06T19:33:22Z

+1

Type specification in cloned /modified python configs dropped as planned
Jenkins tests pass

cmsbuild · 2020-07-06T19:33:50Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-07-07T07:51:13Z

+1

slava77 · 2020-07-07T12:56:11Z

RecoTracker/FinalTrackSelectors/python/MergeTrackCollections_cff.py

    setsToMerge = cms.VPSet( cms.PSet( tLists=cms.vint32(1), pQual=cms.bool(True) )
                             ),


Suggested change

setsToMerge = cms.VPSet( cms.PSet( tLists=cms.vint32(1), pQual=cms.bool(True) )

),

setsToMerge = [ cms.PSet( tLists=cms.vint32(1), pQual=cms.bool(True) )

],

this still works.

I'd think that a nameless tuple like could still work for PSet, like [ ( tLists=cms.vint32(1), pQual=cms.bool(True) ], but this is apparently not supported.
@makortel do you think it would be practical to add?

I don't think ( tLists=cms.vint32(1), pQual=cms.bool(True) ) would be legal syntax

>>> (foo=2) File "<stdin>", line 1 (foo=2) ^ SyntaxError: invalid syntax

So using a keyword argument would require dict(foo=2), which is already used for modifying an existing PSet.

dict does not really apply here because full contents of setsToMerge are replaced (we are not modifying just one of the elements of this array.

Just to reiterate, my question was, if we have support for VInputTag to be initialized from [ "a", b", ("c", "d") ] , can we also get a VPSet to be initialized without listing the element type?

With VInputTag a compact (maybe event too compact) is possible because the meaning of each element of the sequence of strings is known by its position in the sequence. In PSet the element keys are the parameter names, so dict would be a natural "shortcut". But we already use dict for something else, and I'd think anything along ("tlists", cms.vint32(1), "pQual", cms.bool(True)) would be (much) less clear than just using the cms.PSet.

sorry, I meant
instead of
cms.VPSet( cms.PSet( tLists=cms.vint32(1), pQual=cms.bool(True) ) )
to be able to say
[ ( tLists=cms.vint32(1), pQual=cms.bool(True) ) ]
this is not serving the full goal of "drop type specs", but it would at least allow to insert elements to a container of a known type (vector of cms.PSet) without need to explicitly specify the type of the element (cms.PSet)

how about [ { 'tLists' : cms.vint32(1), 'pQual' : cms.bool(True) } ] ?
it would be a valid syntax, but it comes out with raise TypeError("wrong type being extended to container "+self._labelIfAny())

The { 'tLists' : cms.vint32(1), 'pQual' : cms.bool(True) } is a dictionary equivalent to dict(tLists = cms.vint32(1), pQual = cms.bool(True)).

OK, syntactically , this dict( may be more transparent.
I guess from the above the point is that dict doesn't do the right thing here (yet?).

OK, syntactically , this dict( may be more transparent.
I guess from the above the point is that dict doesn't do the right thing here (yet?).

OTOH, the difference to type cms.PSet vs dict( is rather minor; IIRC this is also the only vector type that needs this. So, we may just live with it for now.

OK, syntactically , this dict( may be more transparent.
I guess from the above the point is that dict doesn't do the right thing here (yet?).

We already use dict() to modify an existing PSet. I think using it to denote new PSet in one context would be confusing.

drop type specs where original parameter exists

6bb406b

cmsbuild added this to the CMSSW_11_2_X milestone Jul 6, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jul 6, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 6, 2020

cmsbuild added tests-started and removed tests-pending labels Jul 6, 2020

cmsbuild added tests-approved and removed tests-started labels Jul 6, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jul 6, 2020

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Jul 6, 2020

cmsbuild added orp-approved and removed orp-pending labels Jul 7, 2020

cmsbuild merged commit 7618222 into cms-sw:master Jul 7, 2020

slava77 reviewed Jul 7, 2020

View reviewed changes

jeongeun mentioned this pull request Jul 28, 2020

drop type specs in RecoTracker #30947

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

drop type specs in RecoTracker/{DeDx,FinalTrackSelectors} #30556

drop type specs in RecoTracker/{DeDx,FinalTrackSelectors} #30556

jeongeun commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

perrotta commented Jul 6, 2020

cmsbuild commented Jul 6, 2020 •

edited

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

perrotta commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

silviodonato commented Jul 7, 2020

slava77 Jul 7, 2020

makortel Jul 7, 2020 •

edited

slava77 Jul 7, 2020

makortel Jul 7, 2020

slava77 Jul 7, 2020

slava77 Jul 7, 2020

makortel Jul 7, 2020

slava77 Jul 7, 2020

slava77 Jul 7, 2020

makortel Jul 7, 2020

		setsToMerge = cms.VPSet( cms.PSet( tLists=cms.vint32(1), pQual=cms.bool(True) )
		),

drop type specs in RecoTracker/{DeDx,FinalTrackSelectors} #30556

drop type specs in RecoTracker/{DeDx,FinalTrackSelectors} #30556

Conversation

jeongeun commented Jul 6, 2020

PR description: Update the safer syntax for existing parameter

PR validation:

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

perrotta commented Jul 6, 2020

cmsbuild commented Jul 6, 2020 • edited

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

perrotta commented Jul 6, 2020

cmsbuild commented Jul 6, 2020

silviodonato commented Jul 7, 2020

Choose a reason for hiding this comment

makortel Jul 7, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented Jul 6, 2020 •

edited

makortel Jul 7, 2020 •

edited