PPS: rename directories with misleading names #29037

jan-kaspar · 2020-02-25T15:23:47Z

PR description:

There were two instances of directories with misleading names in the (CT)PPS code tree:

CondFormats/CTPPSReadoutObjects: contained all PPS cond-formats, not just readout,
RecoCTPPS/TotemRPLocal: beyond the code for Si strip detectors (from TOTEM), contained also the code for diamond and UFSD sensors.

We profit from this renaming, to replace CTPPS with PPS in the affected directory names.

In RecoPPS, we put all the local-reconstruction (within a single RP) into a single directory "Local".

This is a technical RP, no changes in the results are expected.

PR validation:

Running

runTheMatrix.py -l limited -j 10 --ibeos

gave

34 33 32 26 16 4 1 1 1 tests passed, 0 0 0 0 0 0 0 0 0 failed

The plots below compare proton reconstruction with CMSSW_11_1_0_pre3 (blue) and this PR (red dashed) - there is no difference as expected.

cmsbuild · 2020-02-25T15:24:21Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-02-25T15:32:06Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29037/13903

This PR adds an extra 424KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/StandardSequences/python/Reconstruction_cff.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983
- File PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py modified in PR(s): PUPPI jets in RECO #28936, New Tau ID data format #28417

cmsbuild · 2020-02-25T15:32:43Z

A new Pull Request was created by @jan-kaspar for master.

It involves the following packages:

CalibPPS/ESProducers
CondCore/CTPPSPlugins
CondCore/Utilities
CondFormats/CTPPSReadoutObjects
CondFormats/PPSObjects
CondTools/CTPPS
Configuration/EventContent
Configuration/StandardSequences
DQM/CTPPS
DQM/Integration
EventFilter/CTPPSRawToDigi
Geometry/VeryForwardGeometry
Geometry/VeryForwardGeometryBuilder
IOMC/EventVertexGenerators
PhysicsTools/PatAlgos
RecoCTPPS/Configuration
RecoCTPPS/PixelLocal
RecoCTPPS/TotemRPLocal
RecoPPS/Configuration
RecoPPS/Local
RecoPPS/ProtonReconstruction
SimPPS/PPSPixelDigiProducer
SimPPS/RPDigiProducer
Validation/CTPPS

The following packages do not have a category, yet:

CondFormats/PPSObjects
RecoPPS/Configuration
RecoPPS/Local
RecoPPS/ProtonReconstruction
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@andrius-k, @schneiml, @ianna, @fioriNTU, @tlampen, @pohsun, @santocch, @perrotta, @civanch, @makortel, @cmsbuild, @davidlange6, @Dr15Jones, @cvuosalo, @mdhildreth, @jfernan2, @tocheng, @slava77, @ggovi, @fabiocos, @kmaeshima, @christopheralanwest, @silviodonato, @franzoni can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @felicepantaleo, @forthommel, @hatakeyamak, @emilbols, @Martin-Grunewald, @ahinzmann, @threus, @peruzzim, @seemasharmafnal, @mmarionncern, @makortel, @smoortga, @jdolen, @ferencek, @GiacomoSguazzoni, @rovere, @VinInn, @jdamgov, @nhanvtran, @gkasieczka, @tocheng, @schoef, @mmusich, @dgulhan, @clelange, @batinkov, @riga, @JyothsnaKomaragiri, @mverzett, @lecriste, @gpetruc, @mariadalfonso, @andrzejnovak this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

perrotta · 2020-02-26T10:38:29Z

please test

cmsbuild · 2020-02-26T10:38:58Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4886/console Started: 2020/02/26 11:43

cmsbuild · 2020-02-26T12:47:47Z

+1
Tested at: f34dc81
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e25e49/4886/summary.html
CMSSW: CMSSW_11_1_X_2020-02-25-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-02-26T12:47:50Z

Comparison job queued.

cmsbuild · 2020-02-26T14:13:22Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e25e49/4886/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e25e49/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e25e49/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e25e49/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e25e49/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e25e49/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-e25e49/23234.0_TTbar_14TeV+TTbar_14TeV_TuneCP5_2026D49_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D49+RecoFullGlobal_2026D49+HARVESTFullGlobal_2026D49

Comparison Summary:

You potentially added 1510 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2679706
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2679386
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

silviodonato · 2020-03-03T21:23:09Z

merge

db: @ggovi
alca: @franzoni @pohsun @tlampen @tocheng @christopheralanwest
analysis: @santocch
dqm: @andrius-k @jfernan2 @kmaeshima @fioriNTU @schneiml
Please have a look. This PR simply renames several directories.

jfernan2 · 2020-03-03T21:43:33Z

+1

cvuosalo · 2020-03-03T21:54:30Z

+geometry

silviodonato · 2020-03-04T09:11:21Z

@jan-kaspar I reverted this PR with #29093 because it is causing many IB errors in CMSSW_11_1_X_2020-03-03-2300 (slc7 amd64 gcc820).
The errors are
class CTPPSBeamParameters found in libCondFormatsCTPPSReadoutObjects.so is already in libCondFormatsPPSObjects.so

@jan-kaspar please open a new PR, I think we need some help from @ggovi to fix the problem with CondFormatsCTPPSReadoutObjects

perrotta · 2020-03-04T09:38:16Z

@jan-kaspar I reverted this PR with #29093 because it is causing many IB errors in CMSSW_11_1_X_2020-03-03-2300 (slc7 amd64 gcc820).
The errors are
class CTPPSBeamParameters found in libCondFormatsCTPPSReadoutObjects.so is already in libCondFormatsPPSObjects.so

@jan-kaspar please open a new PR, I think we need some help from @ggovi to fix the problem with CondFormatsCTPPSReadoutObjects

It is supposedly an issue with root, see above
I would rather ask the advice of the fwk gurus first (@Dr15Jones,@smuzaffar,@makortel)

jan-kaspar · 2020-03-04T10:04:18Z

It is supposedly an issue with root, see above
I would rather ask the advice of the fwk gurus first (@Dr15Jones,@smuzaffar,@makortel)

OK, I am waiting for further advice.

mrodozov · 2020-03-04T11:01:06Z

If you see the IBs, you'll notice the ROOT 620 IB which is also using the master branch of cmssw doesn't have the unit tests breaking and it's also the only one that is a full build (by chance, due to change relevant only to this specific build, this one cms-sw/cmsdist#5619 ).
I'm suspecting there is no problem with this PR but we have to build full builds to dispose of the unit test failures.

perrotta · 2020-03-04T11:10:31Z

If you see the IBs, you'll notice the ROOT 620 IB which is also using the master branch of cmssw doesn't have the unit tests breaking and it's also the only one that is a full build (by chance, due to change relevant only to this specific build, this one cms-sw/cmsdist#5619 ).
I'm suspecting there is no problem with this PR but we have to build full builds to dispose of the unit test failures.

Thank you @mrodozov !
Then, it was reverted too quickly in the IB, and delaying its inclusion will only postpone the problem to pre5.
My impression is that we should rather "revert the revert" and have a full build in the IBs to clean up the unit tests.

mrodozov · 2020-03-04T11:29:25Z

this would be easier (merge it again in cmssw) but I'd rather try to do it manually to confirm so that we don't mess it up further 😅

jan-kaspar · 2020-03-06T07:51:46Z

Thanks everyone! Is any action needed from our side?

perrotta · 2020-03-06T07:59:29Z

@mrodozov @silviodonato : was any attempt made to merge this again manually in the IB and make a full build, as suggested above?

@silviodonato do you plan to eventually "revert the revert", or should we ask @jan-kaspar to prepare an identical PR to be merged after pre4? Please advise.

silviodonato · 2020-03-06T16:28:46Z

@perrotta I made #29093

jan-kaspar · 2020-03-06T16:56:36Z

Thanks @silviodonato !

santocch · 2020-03-08T12:43:48Z

+1

tlampen · 2020-03-09T13:44:26Z

+1

cmsbuild added this to the CMSSW_11_1_X milestone Feb 25, 2020

cmsbuild added alca-pending analysis-pending code-checks-pending comparison-pending db-pending dqm-pending geometry-pending new-package-pending operations-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending labels Feb 25, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 25, 2020

cmsbuild added tests-started and removed tests-pending labels Feb 26, 2020

cmsbuild added tests-approved and removed tests-started labels Feb 26, 2020

cmsbuild removed the comparison-pending label Feb 26, 2020

cmsbuild added orp-approved and removed orp-pending labels Mar 3, 2020

cmsbuild merged commit 4eebb04 into cms-sw:master Mar 3, 2020

cmsbuild added dqm-approved and removed dqm-pending labels Mar 3, 2020

silviodonato mentioned this pull request Mar 4, 2020

Revert "PPS: rename directories with misleading names" #29093

Merged

silviodonato mentioned this pull request Mar 6, 2020

PPS: rename directories with misleading names #29137

Merged

cmsbuild added analysis-approved and removed analysis-pending labels Mar 8, 2020

cmsbuild added alca-approved and removed alca-pending new-package-pending labels Mar 9, 2020

forthommel deleted the pps_rename branch May 3, 2020 10:11

forthommel restored the pps_rename branch May 3, 2020 10:11

jan-kaspar deleted the pps_rename branch September 23, 2020 16:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PPS: rename directories with misleading names #29037

PPS: rename directories with misleading names #29037

jan-kaspar commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

perrotta commented Feb 26, 2020

cmsbuild commented Feb 26, 2020 •

edited

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

silviodonato commented Mar 3, 2020

jfernan2 commented Mar 3, 2020

cvuosalo commented Mar 3, 2020

silviodonato commented Mar 4, 2020

perrotta commented Mar 4, 2020

jan-kaspar commented Mar 4, 2020

mrodozov commented Mar 4, 2020

perrotta commented Mar 4, 2020

mrodozov commented Mar 4, 2020

jan-kaspar commented Mar 6, 2020

perrotta commented Mar 6, 2020

silviodonato commented Mar 6, 2020

jan-kaspar commented Mar 6, 2020

santocch commented Mar 8, 2020

tlampen commented Mar 9, 2020

PPS: rename directories with misleading names #29037

PPS: rename directories with misleading names #29037

Conversation

jan-kaspar commented Feb 25, 2020

PR description:

PR validation:

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

cmsbuild commented Feb 25, 2020

perrotta commented Feb 26, 2020

cmsbuild commented Feb 26, 2020 • edited

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

cmsbuild commented Feb 26, 2020

silviodonato commented Mar 3, 2020

jfernan2 commented Mar 3, 2020

cvuosalo commented Mar 3, 2020

silviodonato commented Mar 4, 2020

perrotta commented Mar 4, 2020

jan-kaspar commented Mar 4, 2020

mrodozov commented Mar 4, 2020

perrotta commented Mar 4, 2020

mrodozov commented Mar 4, 2020

jan-kaspar commented Mar 6, 2020

perrotta commented Mar 6, 2020

silviodonato commented Mar 6, 2020

jan-kaspar commented Mar 6, 2020

santocch commented Mar 8, 2020

tlampen commented Mar 9, 2020

cmsbuild commented Feb 26, 2020 •

edited