New PCL Alignment Threshold object in alignment consumers #38273

dmeuser · 2022-06-07T15:19:54Z

PR description:

This PR is the second step towards using a higher granularity (HG) for the pixel alignment in the PCL, following #38195, where the new alignment threshold object was introduced. In this PR the consumer in the Alignment package are adapted to use the new threshold object.

In detail the following changes have been made:

Change from AlignPCLThresholds to AlignPCLThresholdsHG in the consumer code of the Alignment package
This PR updates the autoCond with GTs containing SiPixelAliThresholdsHG_offline_v0 and SiPixelAliThresholdsHG_express_v0

PR validation:

The PR was tested and validated using runTheMatrix -l 1001.0

Upcoming PRs:

This PR will be followed by one additional PR, which will introduces the changes needed to run and store the HG alignment

New GlobalTags:

The new GTs are

124X_dataRun2_v2
124X_dataRun2_relval_v2
124X_dataRun2_HEfail_v2

124X_dataRun3_v3
124X_dataRun3_relval_v3

124X_dataRun3_Express_frozen_v1
124X_dataRun2_PromptLike_HI_v1
124X_dataRun3_Prompt_frozen_v1
where the frozen GTs are the snapshotter versions of the newly created

124X_dataRun3_Express_v1
124X_dataRun3_Prompt_v1
The snapshot time is
2022-06-09 20:00:00 (UTC)

The diffs

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun2_v1/124X_dataRun2_v2

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun2_relval_v1/124X_dataRun2_relval_v2

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun2_HEfail_v1/124X_dataRun2_HEfail_v2

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_v2/124X_dataRun3_v3

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_relval_v2/124X_dataRun3_relval_v3

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/123X_dataRun3_Express_frozen_v4/124X_dataRun3_Express_frozen_v1

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/123X_dataRun2_PromptLike_HI_v3/124X_dataRun2_PromptLike_HI_v1

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/123X_dataRun3_Prompt_frozen_v4/124X_dataRun3_Prompt_frozen_v1

cc:
@mmusich, @connorpa, @antoniovagnerini, @consuegs

cmsbuild · 2022-06-07T15:26:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38273/30424

This PR adds an extra 48KB to repository

cmsbuild · 2022-06-07T15:26:47Z

A new Pull Request was created by @dmeuser for master.

It involves the following packages:

Alignment/CommonAlignmentProducer (alca)
Alignment/MillePedeAlignmentAlgorithm (alca)

@cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please review it and eventually sign? Thanks.
@pakhotin, @mmusich, @adewit, @tocheng, @tlampen this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

tvami · 2022-06-07T15:30:28Z

type trk

tvami · 2022-06-07T15:37:41Z

Add code to supply the AlignPCLThresholdsHGRcd via ESSource

Ok, let's test this as it is now, and then we should provide this Rcd from the GT in a follow-up. This we can do after the L1T changes have been in the GT (so 2-3 more days)

tvami · 2022-06-07T15:38:05Z

test parameters:

workflows = 1001.0

tvami · 2022-06-07T15:38:12Z

@cmsbuild , please test

mmusich · 2022-06-07T15:41:30Z

Ok, let's test this as it is now, and then we should provide this Rcd from the GT in a follow-up. This we can do after the L1T changes have been in the GT (so 2-3 more days)

No. A new pre-release is needed for inclusion in GT (the record is currently not recognized by the browser).

tvami · 2022-06-07T15:51:11Z

A new pre-release is needed for inclusion in GT (the record is currently not recognized by the browser)

I think that's not true. I can add it with admin rights, this is what I did
AlignPCLThresholdsHGRcd | AlignPCLThresholdsHG
in the record management.

Please confirm that's the correct matching.
If it is please upload the tag to Prod. The Prep DB is buggy so I couldnt do the same there (I'll follow-up with Andres abt the error)

mmusich · 2022-06-07T15:55:11Z

I think that's not true.

this used to be the case in order to protect wrong requests of record on releases that don't support yet new records.
If the behaviour has been changed, this in an unwelcome feature that should be removed.

malbouis · 2022-06-07T16:01:53Z

I think that's not true.

this used to be the case in order to protect wrong requests of record on releases that don't support yet new records. If the behaviour has been changed, this in an unwelcome feature that should be removed.

@mmusich , would you mind expanding on why this is an unwelcome feature?

mmusich · 2022-06-07T16:03:41Z

would you mind expanding on why this is an unwelcome feature?

because it can lead the users (and the conveners unintentionally) to create broken global tags for the intended target queue.

tvami · 2022-06-07T16:32:29Z

OK the pre-release comes out today, so we can wait with this until tomorrow.

cmsbuild · 2022-06-07T19:32:59Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edcbe3/25344/summary.html
COMMIT: 091e7d2
CMSSW: CMSSW_12_5_X_2022-06-07-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38273/25344/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3652078
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3652048
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

mmusich · 2022-06-09T08:24:47Z

@cms-sw/alca-l2 I see that the 12_5_0_pre2 build is ready.
Shall we wait that it becomes propagated to the CondDB tools, re-upload, make new GTs, etc, or can we just proceed with this as is and fix it later? This PR is a blocker for the further (and largest) new developments concerning the HG Alignment PCL.

tvami · 2022-06-09T12:05:54Z

Hi Marco, pre2 is already propagated to the conddb tools.

Please go ahead with uploading the tag to Prod, and we can cut new GTs.
Just a confirmation: this development is not meant to be for 12_3_X (just 12_4_X), correct?

mmusich · 2022-06-09T12:07:47Z

Just a confirmation: this development is not meant to be for 12_3_X (just 12_4_X), correct?

it is meant for at least 12_4_X

tvami · 2022-06-09T12:15:08Z

it is meant for at least 12_4_X

Yes, so that's the grammar that confused me for #38297 (comment)

Ok, let's agree that this is for 12_4_X then

cmsbuild · 2022-06-13T07:33:20Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38273/30539

This PR adds an extra 12KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Adding HZZ electron MVA ID #37429

cmsbuild · 2022-06-13T07:33:41Z

Pull request #38273 was updated. @cmsbuild, @malbouis, @tvami, @yuanchao, @francescobrivio can you please check and sign again.

mmusich · 2022-06-13T07:34:13Z

@cmsbuild, please test

cmsbuild · 2022-06-13T11:34:08Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edcbe3/25473/summary.html
COMMIT: 138bab0
CMSSW: CMSSW_12_5_X_2022-06-12-0000/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38273/25473/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3658678
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3658648
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 49 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

tvami · 2022-06-13T11:48:02Z

+alca

wf testing this development runs fine https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-edcbe3/25473/runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5+ALCAHARVD7+ALCAHARVD8/
@dmeuser please update the PR describtion with the following

This PR updates the autoCond with GTs containing SiPixelAliThresholdsHG_offline_v0 and SiPixelAliThresholdsHG_express_v0

The new GTs are

124X_dataRun2_v2
124X_dataRun2_relval_v2
124X_dataRun2_HEfail_v2

124X_dataRun3_v3
124X_dataRun3_relval_v3

124X_dataRun3_Express_frozen_v1
124X_dataRun2_PromptLike_HI_v1
124X_dataRun3_Prompt_frozen_v1
where the frozen GTs are the snapshotter versions of the newly created

124X_dataRun3_Express_v1
124X_dataRun3_Prompt_v1
The snapshot time is
2022-06-09 20:00:00 (UTC)

The diffs

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun2_v1/124X_dataRun2_v2

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun2_relval_v1/124X_dataRun2_relval_v2

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun2_HEfail_v1/124X_dataRun2_HEfail_v2

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_v2/124X_dataRun3_v3

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/124X_dataRun3_relval_v2/124X_dataRun3_relval_v3

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/123X_dataRun3_Express_frozen_v4/124X_dataRun3_Express_frozen_v1

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/123X_dataRun2_PromptLike_HI_v3/124X_dataRun2_PromptLike_HI_v1

https://cms-conddb.cern.ch/cmsDbBrowser/diff/Prod/gts/123X_dataRun3_Prompt_frozen_v4/124X_dataRun3_Prompt_frozen_v1

cmsbuild · 2022-06-13T11:48:26Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

dmeuser · 2022-06-13T12:55:45Z

* @dmeuser please update the PR describtion with the following

@tvami done

qliphy · 2022-06-13T13:26:40Z

+1

cmsbuild added this to the CMSSW_12_5_X milestone Jun 7, 2022

cmsbuild added alca-pending code-checks-pending orp-pending pending-signatures tests-pending labels Jun 7, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 7, 2022

cmsbuild added the trk label Jun 7, 2022

cmsbuild added tests-started and removed tests-pending labels Jun 7, 2022

cmsbuild added tests-approved and removed tests-started labels Jun 7, 2022

cmsbuild added code-checks-pending tests-pending labels Jun 13, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 13, 2022

cmsbuild added tests-started and removed tests-pending labels Jun 13, 2022

cmsbuild added tests-approved and removed tests-started labels Jun 13, 2022

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Jun 13, 2022

cmsbuild added orp-approved and removed orp-pending labels Jun 13, 2022

cmsbuild merged commit b61dc7e into cms-sw:master Jun 13, 2022

cmsbuild mentioned this pull request Jun 13, 2022

Convert xml.gz files to .root files cms-data/RecoEgamma-ElectronIdentification#26

Merged

dmeuser mentioned this pull request Jun 14, 2022

Fix size of arrays in MillePedeDQMModule to store new HG thresholds #38367

Merged

mmusich mentioned this pull request Jun 14, 2022

New PCL Alignment Threshold object and record for HG PCL Alignment [12.4.X] #38371

Merged

smuzaffar mentioned this pull request Jun 15, 2022

[CLANG_X] Segmentation violation in MillePedeDQMModule::dqmEndJob #38364

Closed

This was referenced Jun 20, 2022

Update L1T menu tag in MC and relval GTs #38438

Merged

[12_4_X] Update L1T menu tag in MC and relval GTs #38439

Merged

dmeuser mentioned this pull request Jun 21, 2022

New high granularity pixel alignment for the PCL #38449

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New PCL Alignment Threshold object in alignment consumers #38273

New PCL Alignment Threshold object in alignment consumers #38273

dmeuser commented Jun 7, 2022 •

edited

cmsbuild commented Jun 7, 2022

cmsbuild commented Jun 7, 2022

tvami commented Jun 7, 2022

tvami commented Jun 7, 2022

tvami commented Jun 7, 2022

tvami commented Jun 7, 2022

mmusich commented Jun 7, 2022 •

edited

tvami commented Jun 7, 2022 •

edited

mmusich commented Jun 7, 2022 •

edited

malbouis commented Jun 7, 2022

mmusich commented Jun 7, 2022

tvami commented Jun 7, 2022

cmsbuild commented Jun 7, 2022

mmusich commented Jun 9, 2022

tvami commented Jun 9, 2022

mmusich commented Jun 9, 2022

tvami commented Jun 9, 2022

cmsbuild commented Jun 13, 2022

cmsbuild commented Jun 13, 2022

mmusich commented Jun 13, 2022

cmsbuild commented Jun 13, 2022

tvami commented Jun 13, 2022

cmsbuild commented Jun 13, 2022

dmeuser commented Jun 13, 2022

qliphy commented Jun 13, 2022

New PCL Alignment Threshold object in alignment consumers #38273

New PCL Alignment Threshold object in alignment consumers #38273

Conversation

dmeuser commented Jun 7, 2022 • edited

PR description:

PR validation:

Upcoming PRs:

New GlobalTags:

cmsbuild commented Jun 7, 2022

cmsbuild commented Jun 7, 2022

tvami commented Jun 7, 2022

tvami commented Jun 7, 2022

tvami commented Jun 7, 2022

tvami commented Jun 7, 2022

mmusich commented Jun 7, 2022 • edited

tvami commented Jun 7, 2022 • edited

mmusich commented Jun 7, 2022 • edited

malbouis commented Jun 7, 2022

mmusich commented Jun 7, 2022

tvami commented Jun 7, 2022

cmsbuild commented Jun 7, 2022

Comparison Summary

mmusich commented Jun 9, 2022

tvami commented Jun 9, 2022

mmusich commented Jun 9, 2022

tvami commented Jun 9, 2022

cmsbuild commented Jun 13, 2022

cmsbuild commented Jun 13, 2022

mmusich commented Jun 13, 2022

cmsbuild commented Jun 13, 2022

Comparison Summary

tvami commented Jun 13, 2022

cmsbuild commented Jun 13, 2022

dmeuser commented Jun 13, 2022

qliphy commented Jun 13, 2022

dmeuser commented Jun 7, 2022 •

edited

mmusich commented Jun 7, 2022 •

edited

tvami commented Jun 7, 2022 •

edited

mmusich commented Jun 7, 2022 •

edited