Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLUE3D Bug Fixes #36857

Merged
merged 2 commits into from Feb 8, 2022
Merged

CLUE3D Bug Fixes #36857

merged 2 commits into from Feb 8, 2022

Conversation

rovere
Copy link
Contributor

@rovere rovere commented Feb 1, 2022

PR description:

Fix bug related to unsigned int subtraction. Also, add a customize
python function to allow to save in output only the data-products that
are needed to run the HGCAL (local) reconstruction from. Finally,
increase the search window in eta to 2, for all eta values.

PR validation:

Tested using runTheMatrix workflows with clue3D enabled.

Fix bug related to unsigned int subtraction. Also, add a customize
python function to allow to save in output only the data-products that
are needed to run the HGCAL (local) reconstruction from. Finally,
increase the search window in eta to 2, for all eta values.
@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 1, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36857/28098

  • This PR adds an extra 24KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 1, 2022

A new Pull Request was created by @rovere (Marco Rovere) for master.

It involves the following packages:

  • RecoHGCal/Configuration (upgrade, reconstruction)
  • RecoHGCal/TICL (upgrade, reconstruction)

@clacaputo, @cmsbuild, @AdrianoDee, @srimanob, @slava77, @jpata can you please review it and eventually sign? Thanks.
@felicepantaleo, @apsallid, @sobhatta, @lecriste, @hatakeyamak, @ebrondol this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@rovere
Copy link
Contributor Author

rovere commented Feb 1, 2022

test parameters:

  • workflow = 35094.201,35096.201
  • relvals_opt = --what upgrade,standard,highstats,pileup,generator,extendedgen,production,ged,machine,premix

@rovere
Copy link
Contributor Author

rovere commented Feb 1, 2022

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 2, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0b70d6/22135/summary.html
COMMIT: f9b21e5
CMSSW: CMSSW_12_3_X_2022-02-01-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36857/22135/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0b70d6/35094.201_CloseByPGun_CE_E_Front_300um+2026D77_ticl_clue3D+CE_E_Front_300um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0b70d6/35096.201_CloseByPGun_CE_E_Front_120um+2026D77_ticl_clue3D+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 2 differences found in the comparisons
  • DQMHistoTests: Total files compared: 43
  • DQMHistoTests: Total histograms compared: 3449612
  • DQMHistoTests: Total failures: 5
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 3449584
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.004 KiB( 42 files compared)
  • DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
  • Checked 181 log files, 42 edm output root files, 43 DQM output files
  • TriggerResults: no differences found

int etaWindow = onSameLayer ? 2 : 1;
int phiWindow = onSameLayer ? 2 : 1;
const int etaWindow = 2;
const int phiWindow = 2;
Copy link
Contributor

@AdrianoDee AdrianoDee Feb 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for my knowledge, I most probably have missed some passages: why this was restricted to 1 for layerClusters on different layers?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ciao @AdrianoDee, the original motivation for having 2 different cuts, within and outside of the layer, is related to the fact that we are clustering in 3D LayerClusters. Therefore there should be no need to further cluster them within every single layer or, if you want, you would need to search a little farther away to find some other clusters to collect on the same layer. The move to 2 was taken from the CA experience: we notice that, especially at high eta values, a single bin in eta,phi can be quite restrictive. Hence to move to just use one value, 2, to cover them all. When we reach the point of including CLUE3D in the reconstruction, we will need, possibly, some further tuning.

@@ -718,7 +719,7 @@ void PatternRecognitionbyCLUE3D<TILES>::fillPSetDescription(edm::ParameterSetDes
iDesc.add<bool>("densityOnSameLayer", false);
iDesc.add<double>("criticalEtaPhiDistance", 0.035);
iDesc.add<double>("outlierMultiplier", 2);
iDesc.add<int>("minNumLayerCluster", 5)->setComment("Not Inclusive");
iDesc.add<int>("minNumLayerCluster", 2)->setComment("Not Inclusive");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(As above, I have surely missed something). Why is this going from 5 to 2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ciao @AdrianoDee in this case there's no real reason. The idea, in this case, is that the selection will be done on top of this and will likely use more advanced ML techniques. For this reason, it could be better to allow more input to possibly gather a little more in output. I hope this is clear.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thanks @rovere for all the details (here and above).

@AdrianoDee
Copy link
Contributor

+Upgrade
Wf running smooth. New event content defined. Minor changes.

@rovere
Copy link
Contributor Author

rovere commented Feb 3, 2022

test parameters:

  • workflow = 35094.201,35096.201
  • relvals_opt = --what upgrade,standard,highstats,pileup,generator,extendedgen,production,ged,machine,premix

@rovere
Copy link
Contributor Author

rovere commented Feb 3, 2022

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 3, 2022

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36857/28140

  • This PR adds an extra 12KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 3, 2022

Pull request #36857 was updated. @jpata, @AdrianoDee, @srimanob, @clacaputo, @slava77 can you please check and sign again.

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 3, 2022

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0b70d6/22201/summary.html
COMMIT: 03068ab
CMSSW: CMSSW_12_3_X_2022-02-03-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/36857/22201/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0b70d6/35094.201_CloseByPGun_CE_E_Front_300um+2026D77_ticl_clue3D+CE_E_Front_300um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0b70d6/35096.201_CloseByPGun_CE_E_Front_120um+2026D77_ticl_clue3D+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0b70d6/39434.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0b70d6/39496.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
  • /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-0b70d6/39500.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 6 differences found in the comparisons
  • DQMHistoTests: Total files compared: 46
  • DQMHistoTests: Total histograms compared: 3765022
  • DQMHistoTests: Total failures: 13
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 3764986
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.004 KiB( 45 files compared)
  • DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
  • Checked 193 log files, 42 edm output root files, 46 DQM output files
  • TriggerResults: no differences found

@AdrianoDee
Copy link
Contributor

+Upgrade
(Just added associator w.r.t. to latest sign off)

@clacaputo
Copy link
Contributor

Hi @rovere, I've tested 35094.201,35096.201 locally, with and without the bug fix and I don't find any differences in reco products. Is it expected, considering the change in the eta window?

@rovere
Copy link
Contributor Author

rovere commented Feb 8, 2022

Thanks @clacaputo for the additional checks. The differences will not be dramatic and will impact mainly the tracksters in the very forward region (eta 2.5 or higher, more or less). The samples you have used generate particles at eta ~1.6 and ~2.45 respectively for 35094.201 and 35096.201. I do expect no differences at all for the former and very minor, if at all, for the latter.

@clacaputo
Copy link
Contributor

@rovere, thanks a lot for the clarification

@clacaputo
Copy link
Contributor

+reconstruction

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 8, 2022

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

auto lastLayerPerSide = static_cast<unsigned int>(rhtools_.lastLayer(false)) - 1;
unsigned int minLayer = 0;
unsigned int maxLayer = 2 * lastLayerPerSide - 1;
auto lastLayerPerSide = static_cast<int>(rhtools_.lastLayer(false));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the same as previous lastLayerPerSide, and maxLayer as well as a consequence: please confirm that this is wanted

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ciao @perrotta that's correct, the -1 was a bug as well.

@perrotta
Copy link
Contributor

perrotta commented Feb 8, 2022

+1

@cmsbuild cmsbuild merged commit e798919 into cms-sw:master Feb 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants