Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean DQMOffline/MuonDPG/BuildFile.xml and remove UBSAN comments #36855

Merged
merged 1 commit into from Feb 6, 2022
Merged

Clean DQMOffline/MuonDPG/BuildFile.xml and remove UBSAN comments #36855

merged 1 commit into from Feb 6, 2022

Conversation

guitargeek
Copy link
Contributor

@guitargeek guitargeek commented Feb 1, 2022

PR description:

Note: this should be tested like please test for CMSSW_12_3_UBSAN_X (see #36834 (comment))

At a few places, there are comments in Build files that a given
dependency is "needed for UBSAN builds". I'd argue that this these
comments are confusing an should be removed: if a package has a
dependency it's because headers from that dependency are included, so no
need to argue with the UBSAN builds.

Both in the CalibTracker/SiStripHitEfficiency and
DQM/TrackerRemapper BuildFiles, the packages indeed include headers
from DataFormats/Scalers, so arguing with UBSAN builds is completely
redundant.

A good example for why these comments are confusing was the BuildFile in
DQMOffline/MuonDPG that package doesn't actually depend on
DataFormats/Scalers as far as I can tell, but because the BuildFile
was copy-pasted from DQM/TrackerRemapper it included the comment about
UBSAN builds which was not even applicable. That's why I didn't clean
that BuildFile in #36834, but instead make this separate commit to
elaborate the arguments.

PR validation:

CMSSW compiles.

if this PR is a backport please specify the original PR and why you need to backport that PR:

No backport intended.

At a few places, there are comments in Build files that a given
dependency is "needed for UBSAN builds. I'd argue that this these
comments are confusing an should be removed: if a package has a
dependency it's because headers from that dependency are included, so no
need to argue with the UBSAN builds.

Both in the `CalibTracker/SiStripHitEfficiency`  and
`DQM/TrackerRemapper` BuildFiles, the packages indeed include headers
from `DataFormats/Scalers`, so arguing with UBSAN builds is completely
redundant.

A good example for why these comments are confusing was the BuildFile in
`DQMOffline/MuonDPG` that package doesn't actually depend on
`DataFormats/Scalers` as far as I can tell, but because the BuildFile
was copy-pasted from `DQM/TrackerRemapper` it included the comment about
UBSAN builds which was not even applicable. That's why I didn't clean
that BuildFile in #36834, but instead make this separate commit to
elaborate the arguments.
@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 1, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36855/28096

  • This PR adds an extra 16KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 1, 2022

A new Pull Request was created by @guitargeek (Jonas Rembser) for master.

It involves the following packages:

  • CalibTracker/SiStripHitEfficiency (alca)
  • DQM/TrackerRemapper (dqm)
  • DQMOffline/MuonDPG (dqm)
  • Geometry/MTDNumberingBuilder (geometry, upgrade)

@malbouis, @civanch, @yuanchao, @makortel, @cvuosalo, @emanueleusai, @ianna, @mdhildreth, @tvami, @cmsbuild, @AdrianoDee, @Dr15Jones, @jfernan2, @ahmad3213, @pmandrik, @srimanob, @francescobrivio, @pbo0, @rvenditti can you please review it and eventually sign? Thanks.
@echabert, @pieterdavid, @robervalwalsh, @bsunanda, @fioriNTU, @threus, @cericeci, @hdelanno, @jhgoh, @HuguesBrun, @ptcox, @trocino, @rociovilar, @sscruz, @abbiendi, @barvic, @tocheng, @bellan, @jandrea, @idebruyn, @mmusich, @fabiocos, @gbenelli, @Fedespring, @calderona this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@tvami
Copy link
Contributor

tvami commented Feb 1, 2022

@cmsbuild, please test for CMSSW_12_3_UBSAN_X

@mmusich
Copy link
Contributor

mmusich commented Feb 1, 2022

These comments were added by @dan131riley in #34102, presumably on behalf of the core framework group. Perhaps the author could comment on the intentions

@dan131riley
Copy link

dan131riley commented Feb 1, 2022

UBSAN builds sometimes need typeinfo for types that are forward declared. The places where I added dependencies with the comment about UBSAN builds, those modules built fine with forward references without those dependencies declared, except in UBSAN builds. If those modules now have an explicit dependency on the types that they previously just forward declared, then removing the UBSAN comment is fine, but to be clear--there are cases where UBSAN needs to see the typeinfo for classes that have only been forward declared, not included.

@guitargeek
Copy link
Contributor Author

guitargeek commented Feb 1, 2022

Hi @dan131riley, thanks for clarifying and supporting the case that the comments can be removed if there are direct dependencies! Just for the record, the direct dependencies are here:
https://github.com/cms-sw/cmssw/blob/master/CalibTracker/SiStripHitEfficiency/interface/HitEff.h#L33
https://github.com/cms-sw/cmssw/blob/master/DQM/TrackerRemapper/interface/SiStripTkMaps.h#L15
https://github.com/cms-sw/cmssw/blob/master/Geometry/MTDNumberingBuilder/interface/MTDTopology.h#L8

In DQMOffline/MuonDPG there is no direct dependency on DataFormats/Scalers, but again the UBSAN comment there was not added by you there but supposedly erroneously by someone else when copy-pasting from the DQM/TrackerRemapper BuildFile, which is the whole point of this PR (it took me a while to see that this was just from copy-paste and not because there was actually a forward declaration of a class from DataFormats/Scalers, and even now I'm still not sure... the UBSAN build by the bot will tell if I was right or not).

@mmusich
Copy link
Contributor

mmusich commented Feb 2, 2022

Just for the record, the direct dependencies are here

Just wondering when these developed (supposedly after the comments were added)? were they hidden by other includes that got eventually cleaned?

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 2, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61d5d6/22134/summary.html
COMMIT: 9c671f2
CMSSW: CMSSW_12_3_UBSAN_X_2022-02-01-1100/slc7_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36855/22134/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61d5d6/22134/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61d5d6/22134/git-merge-result

Comparison Summary

Summary:

  • You potentially added 121130 lines to the logs
  • Reco comparison results: 66230 differences found in the comparisons
  • DQMHistoTests: Total files compared: 43
  • DQMHistoTests: Total histograms compared: 3449612
  • DQMHistoTests: Total failures: 361490
  • DQMHistoTests: Total nulls: 65
  • DQMHistoTests: Total successes: 3088035
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -1.061 KiB( 42 files compared)
  • DQMHistoSizes: changed ( 10224.0 ): -0.117 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 11834.0 ): -1.235 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 250202.181 ): 0.533 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 25202.0 ): -0.246 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
  • Checked 181 log files, 42 edm output root files, 43 DQM output files
  • TriggerResults: found differences in 16 / 42 workflows

@tvami
Copy link
Contributor

tvami commented Feb 2, 2022

+alca

  • technical PR
  • UBSAN is successfully built

@guitargeek
Copy link
Contributor Author

Just wondering when these developed (supposedly after the comments were added)? were they hidden by other includes that got eventually cleaned?

Some of them were added later, some of them were actually already there before. I didn't investigate the history so much because what matters is that these includes are there now, right?

@mmusich
Copy link
Contributor

mmusich commented Feb 2, 2022

I didn't investigate the history so much because what matters is that these includes are there now, right?

Well, just trying to see why the ones that were there before did compile outside of UBSAN_X builds, because if they didn't the author of many of these BuildFiles (me) would have noticed the missing dependency... but fine

@guitargeek
Copy link
Contributor Author

Ah, I see what you mean! I also don't know exactly why the UBSAN builds notice these missing dependencies while the regular builds don't

@guitargeek
Copy link
Contributor Author

Thanks a lot for the explanation!

@jfernan2
Copy link
Contributor

jfernan2 commented Feb 3, 2022

please test for CMSSW_12_3_UBSAN_X
To get a clean DQM comparison without extra commits not from this PR

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 5, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61d5d6/22183/summary.html
COMMIT: 9c671f2
CMSSW: CMSSW_12_3_UBSAN_X_2022-02-02-1100/slc7_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36855/22183/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 130879 lines to the logs
  • Reco comparison results: 69520 differences found in the comparisons
  • DQMHistoTests: Total files compared: 46
  • DQMHistoTests: Total histograms compared: 3765022
  • DQMHistoTests: Total failures: 269432
  • DQMHistoTests: Total nulls: 112
  • DQMHistoTests: Total successes: 3495456
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.421 KiB( 45 files compared)
  • DQMHistoSizes: changed ( 10224.0 ): -0.117 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 11834.0 ): -0.591 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 250202.181 ): 0.533 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 25202.0 ): -0.246 KiB SiStrip/MechanicalView
  • Checked 193 log files, 42 edm output root files, 46 DQM output files
  • TriggerResults: found differences in 16 / 45 workflows

@civanch
Copy link
Contributor

civanch commented Feb 5, 2022

+1

@jfernan2
Copy link
Contributor

jfernan2 commented Feb 5, 2022

+1

@srimanob
Copy link
Contributor

srimanob commented Feb 5, 2022

+Upgrade

Technical PR.

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 5, 2022

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

@perrotta
Copy link
Contributor

perrotta commented Feb 6, 2022

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants