Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MF standardized regression tests + fix some loose ends in MF #28642

Closed
wants to merge 3 commits into from

Conversation

namapane
Copy link
Contributor

PR description:

PR validation:

Ran regression tests against earlier regression files (produced in past CMSSW releases) for all possible combinations of era, current, and MF producer type.

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28642/13197

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@namapane
Copy link
Contributor Author

SimPPS/PPSPixelDigiProducer/test/FromGun2DigiAnal_.py is indeed being removed also in #28190, I was not aware of that since it was not mentioned in the original discussion in #28280. I am not sure how to handle this, should I make a commit where I put back this file in the branch I used for this PR? Thanks, N.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28642/13201

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @namapane (Nicola Amapane) for master.

It involves the following packages:

DetectorDescription/DDCMS
MagneticField/Engine
MagneticField/GeomBuilder
SimPPS/PPSPixelDigiProducer

@perrotta, @civanch, @Dr15Jones, @cvuosalo, @ianna, @mdhildreth, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@vargasa this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Dec 17, 2019

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Dec 17, 2019

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4039/console Started: 2019/12/17 22:30

@cmsbuild
Copy link
Contributor

+1
Tested at: 6a02c53
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7edade/4039/summary.html
CMSSW: CMSSW_11_1_X_2019-12-17-1100
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

Comparison job queued.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7edade/4039/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2798405
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2798063
  • DQMHistoTests: Total skipped: 341
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

# Always use the standard sequence Configuration.StandardSequences.MagneticField_cff


DDDetectorESProducer = cms.ESSource("DDDetectorESProducer",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this will not work in the standard setup due to a conflict with DDDetectorESProducer DD4hep_GeometrySim_cff.
perhaps add MF or smth for a different name?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, will do

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funny, enough it does not work if I rename it to eg

process.DDDetectorMFESProducer = cms.ESSource("DDDetectorESProducer", ...

I then get:

An exception of category 'NoProxyException' occurred while
[0] Processing Event run: 1 lumi: 1 event: 1 stream: 0
[1] Running path 'p'
[2] Calling method for module testMagGeometryAnalyzer/'test'
[3] Using EventSetup component DD4hep_VolumeBasedMagneticFieldESProducer/'MagneticFieldESProducer' to make data MagneticField/'' in record Id
[4] Using EventSetup component DDCompactViewMFESProducer/'' to make data DDCompactView/'magfield' in record IdealMagneticFieldRecord
Exception Message:
No data of type "DDDetector" with label "magfield" in record "IdealMagneticFieldRecord"

It can be quickly reproduced editing

process.DDDetectorESProducer = cms.ESSource("DDDetectorESProducer",
.

@cvuosalo, @ianna, any idea of why this happens and how to avoid this?

@slava77 I will go on with the other fixes (in a new PR) while this is getting understood.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused in what's failing and what is edited.
volumeBasedMagneticField_dd4hep_160812_cfi.py is currently included only in regression.py, but it runs only testMagneticField, while the error message is coming from testMagGeometryAnalyzer.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to apply the modification in cmssw/MagneticField/GeomBuilder/test/python/testMagGeometry.py first, as mentioned in my message above. This contains a (stripped-down) copy of the same config, for the purpose of testing the geometry.
Of course I need to fix volumeBasedMagneticField_dd4hep_160812_cfi.py as well, but the problem would be the same.

label = cms.untracked.string(''),
attribute = cms.string('magfield'),
value = cms.string('magfield'),
paramLabel = cms.string('parametrizedField'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like only DDDetector, attribute, and value are different from volumeBasedMagneticField_160812_cfi.
The ParametrizedMagneticFieldProducer above is also the same.
If this is supposed to stay for more than a week or so, it seems reasonable to refactor the common parts in volumeBasedMagneticField_160812_common_cfi

 ParametrizedMagneticFieldProducer = cms.ESProducer("ParametrizedMagneticFieldProducer",
...

VolumeBasedMagneticFieldCommon = cms.PSet (
#common parts here 
)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is supposed to stay until DD4Hep becomes the default and DDD is dropped, which I guess is not happening in a week or so.
I have a slight preference for keeping the minimum number of cfi files since they quickly get confusing (one more is to be added for the DD4Hep producer from DB), but OK if you insist...

BTW could you suggest what I should do with the file that I removed (FromGun2DigiAnal_.py) which is also being addressed in another PR (see above)? Thanks.

@slava77
Copy link
Contributor

slava77 commented Dec 19, 2019

SimPPS/PPSPixelDigiProducer/test/FromGun2DigiAnal_.py is indeed being removed also in #28190, I was not aware of that since it was not mentioned in the original discussion in #28280. I am not sure how to handle this, should I make a commit where I put back this file in the branch I used for this PR? Thanks, N.

I didn't look inside: does this file require modifications to be consistent with this PR if it is not removed?
If yes, I'd certainty prefer it's removed already in this PR (the other PR is converging rather slowly and we should not wait here).

If no, I have less strong opinion and can leave it to a coin-toss for removing here and possible rebase need in #28190 or dropping its removal here .

@namapane
Copy link
Contributor Author

It just need to be removed (nothing else affected), which I did here since I did not hear about anything happening after the discussion in #28280. I think it belongs better to #28190, so given it is removed there I would drop the removal here. My question is just how to do this in git, should I just push a commit where I re-add back the original file, or should I use some arcane git spell?

@slava77
Copy link
Contributor

slava77 commented Dec 19, 2019

should I just push a commit where I re-add back the original file, or should I use some arcane git spell?

it's better to rebase interactively and remove the commit that removed the file.

@namapane
Copy link
Contributor Author

Sorry but I have no idea no of how to do this practically. Can you send me more detailed instructions? Otherwise I can close this PR and make a new one...

@slava77
Copy link
Contributor

slava77 commented Dec 19, 2019

Sorry but I have no idea no of how to do this practically. Can you send me more detailed instructions? Otherwise I can close this PR and make a new one...

I see that you removed the file in a commit that includes other changes.
The cleanest is to remake a PR; but it's still acceptable to just re-add the file.

@namapane namapane mentioned this pull request Dec 20, 2019
@namapane
Copy link
Contributor Author

Replaced by #28665.

@namapane namapane closed this Dec 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants