Configure DtCalib AlCaReco for heavy Ion 2018 #25004

tocheng · 2018-10-25T19:49:47Z

Greetings,

The PR is to add DtCalibHI into the AlCaReco stream.
The DtCalib in the AlCaReco stream is replaced by sequence in DtCalibHI using the pp_on_AA_2018 era.
Also the DtCalibOutput is replaced by the DtCalibHIOut using the pp_on_AA_2018 era.

The configuration is checked using
python $CMSSW_BASE/src/Configuration/DataProcessing/test/RunExpressProcessing.py --scenario ppEra_Run2_2018_pp_on_AA --global-tag 103X_dataRun2_Express_v1 --lfn /store/whatever --fevt --dqmio --alcareco DtCalib

After changing pickle to python config,
one can see that the DtCalib path and output is replaced by the path and output in DtCalibHI

…alibHI

cmsbuild · 2018-10-25T19:50:08Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-10-25T19:52:21Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-25004/6982

cmsbuild · 2018-10-25T19:52:37Z

A new Pull Request was created by @tocheng (Tongguang) for master.

It involves the following packages:

CalibMuon/DTCalibration

@cmsbuild, @franzoni, @pohsun, @tocheng, @lpernie can you please review it and eventually sign? Thanks.
@mmusich, @tocheng this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

tocheng · 2018-10-25T19:53:50Z

@cmsbuild please test

cmsbuild · 2018-10-25T19:54:07Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/31275/console Started: 2018/10/25 21:54

mmusich · 2018-10-25T20:01:59Z

CalibMuon/DTCalibration/python/ALCARECODtCalib_Output_cff.py

+
+OutALCARECODtCalibHI = copy.deepcopy(OutALCARECODtCalib_noDrop)
+OutALCARECODtCalibHI.outputCommands.insert(0, "drop *")
+OutALCARECODtCalibHI.outputCommands.append("keep *_hiSelectedVertex_*_*")


Hi @tocheng I am wondering is this event product available with thepp_on_AA_2018 era customisations?
@mandrenguyen

@mmusich Thanks for pointing it out. I checked the configuration generated using DP config with pp_on_AA scenario, the hiSelectedVertex is not in the sequence. It should be the module in heavyIon typed reconstruction. I can remove it after the test is done.

mmusich · 2018-10-25T21:04:29Z

Hello @fabiocos, @slava77,
while this PR runs fine in the last IB of 10.3.X using:

python $CMSSW_BASE/src/Configuration/DataProcessing/test/RunExpressProcessing.py --scenario ppEra_Run2_2018_pp_on_AA --global-tag 103X_dataRun2_Express_v1 --lfn /store/t0streamer/Data/HIExpress/000/325/112/run325112_ls0013_streamHIExpress_StorageManager.dat --fevt --dqmio --alcareco DtCalib

and then changing the event source with:

process.source = cms.Source("NewEventStreamFileReader"

while doing the same in 10.4.X I notice I incur in this error:

----- Begin Fatal Exception 25-Oct-2018 22:56:57 CEST-----------------------
An exception of category 'ScheduleExecutionFailure' occurred while
   [0] Calling beginJob
Exception Message:
Unrunnable schedule
Module run order problem found: 
hiConformalPixelTracks after hiConformalPixelTracksPhase1Filter [path reconstruction_step], hiConformalPixelTracksPhase1Filter after pixelFitterByConformalMappingAndLine [path reconstruction_step], pixelFitterByConformalMappingAndLine after hiConformalPixelTracksPhase1HitQuadrupletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitQuadrupletsCA after hiConformalPixelTracksPhase1HitDoubletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitDoubletsCA after hiConformalPixelTracksPhase1SeedLayers [path reconstruction_step], hiConformalPixelTracksPhase1SeedLayers after hiConformalPixelTracksPhase1TrackingRegions [path reconstruction_step], hiConformalPixelTracksPhase1TrackingRegions after hiClusterCompatibility [path reconstruction_step], hiClusterCompatibility after hiCentrality [path reconstruction_step], hiCentrality consumes hiConformalPixelTracks
 Running in the threaded framework would lead to indeterminate results.
 Please change order of modules in mentioned Path(s) to avoid inconsistent module ordering.
----- End Fatal Exception -------------------------------------------------

the same error is also obtained in a vanilla CMSSW_10_4_X_2018-10-25-1100.
Is there any substantial difference between the tip of 10.3.X and the tip of 10.4.X that can cause this?

mandrenguyen · 2018-10-25T21:17:21Z

No, hiSelectedVertex is not available. It’s from the old HeavyIons scenario. With the pp_on_AA era, we use offlinePrimaryVertices

…

On Oct 25, 2018, at 23:04, Marco Musich ***@***.***> wrote: Hello @fabiocos, @slava77, while this PR runs fine in the last IB of 10.3.X using: python $CMSSW_BASE/src/Configuration/DataProcessing/test/RunExpressProcessing.py --scenario ppEra_Run2_2018_pp_on_AA --global-tag 103X_dataRun2_Express_v1 --lfn /store/t0streamer/Data/HIExpress/000/325/112/run325112_ls0013_streamHIExpress_StorageManager.dat --fevt --dqmio --alcareco DtCalib and then changing the the event source with: process.source = cms.Source("NewEventStreamFileReader" while doing the same in 10.4.X I notice I incur in this error: ----- Begin Fatal Exception 25-Oct-2018 22:56:57 CEST----------------------- An exception of category 'ScheduleExecutionFailure' occurred while [0] Calling beginJob Exception Message: Unrunnable schedule Module run order problem found: hiConformalPixelTracks after hiConformalPixelTracksPhase1Filter [path reconstruction_step], hiConformalPixelTracksPhase1Filter after pixelFitterByConformalMappingAndLine [path reconstruction_step], pixelFitterByConformalMappingAndLine after hiConformalPixelTracksPhase1HitQuadrupletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitQuadrupletsCA after hiConformalPixelTracksPhase1HitDoubletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitDoubletsCA after hiConformalPixelTracksPhase1SeedLayers [path reconstruction_step], hiConformalPixelTracksPhase1SeedLayers after hiConformalPixelTracksPhase1TrackingRegions [path reconstruction_step], hiConformalPixelTracksPhase1TrackingRegions after hiClusterCompatibility [path reconstruction_step], hiClusterCompatibility after hiCentrality [path reconstruction_step], hiCentrality consumes hiConformalPixelTracks Running in the threaded framework would lead to indeterminate results. Please change order of modules in mentioned Path(s) to avoid inconsistent module ordering. ----- End Fatal Exception ------------------------------------------------- the same error is also obtained in a vanilla CMSSW_10_4_X_2018-10-25-1100. Is there any substantial difference between the tip of 10.3.X and the tip of 10.4.X that can cause this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

cmsbuild · 2018-10-25T22:16:07Z

+1
Tested at: 29bc4d6
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25004/31275/summary.html

cmsbuild · 2018-10-25T22:16:09Z

Comparison job queued.

cmsbuild · 2018-10-25T22:53:59Z

The code-checks are being triggered in jenkins.

cmsbuild · 2018-10-25T22:56:10Z

Pull request #25004 was updated. @cmsbuild, @franzoni, @pohsun, @tocheng, @lpernie can you please check and sign again.

tocheng · 2018-10-25T22:57:16Z

@cmsbuild please test

cmsbuild · 2018-10-25T22:57:33Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/31279/console Started: 2018/10/26 00:58

cmsbuild · 2018-10-26T01:35:18Z

+1
Tested at: dfc339c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25004/31279/summary.html

cmsbuild · 2018-10-26T01:35:21Z

Comparison job queued.

cmsbuild · 2018-10-26T02:58:32Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-25004/31279/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 32
DQMHistoTests: Total histograms compared: 2994843
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2994645
DQMHistoTests: Total skipped: 197
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
Checked 134 log files, 14 edm output root files, 32 DQM output files

tocheng · 2018-10-26T03:48:00Z

+1

cmsbuild · 2018-10-26T03:48:15Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2018-10-26T04:47:12Z

@tocheng did you verify it works? If I use your recipe

python $CMSSW_BASE/src/Configuration/DataProcessing/test/RunExpressProcessing.py --scenario ppEra_Run2_2018_pp_on_AA --global-tag 103X_dataRun2_Express_v1 --lfn /store/whatever --fevt --dqmio --alcareco DtCalib

and replace the input with

process.source = cms.Source("NewEventStreamFileReader",
fileNames = cms.untracked.vstring('/store/t0streamer/Data/HIPhysicsCommissioning/000/325/112/run325112_ls0006_streamHIPhysicsCommissioning_StorageManage
r.dat')
)

I get the following problem:

----- Begin Fatal Exception 26-Oct-2018 06:44:50 CEST-----------------------
An exception of category 'ScheduleExecutionFailure' occurred while
[0] Calling beginJob
Exception Message:
Unrunnable schedule
Module run order problem found:
hiConformalPixelTracks after hiConformalPixelTracksPhase1Filter [path reconstruction_step], hiConformalPixelTracksPhase1Filter after pixelFitterByConformalMappingAndLine [path reconstruction_step], pixelFitterByConformalMappingAndLine after hiConformalPixelTracksPhase1HitQuadrupletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitQuadrupletsCA after hiConformalPixelTracksPhase1HitDoubletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitDoubletsCA after hiConformalPixelTracksPhase1SeedLayers [path reconstruction_step], hiConformalPixelTracksPhase1SeedLayers after hiConformalPixelTracksPhase1TrackingRegions [path reconstruction_step], hiConformalPixelTracksPhase1TrackingRegions after hiClusterCompatibility [path reconstruction_step], hiClusterCompatibility after hiCentrality [path reconstruction_step], hiCentrality consumes hiConformalPixelTracks
Running in the threaded framework would lead to indeterminate results.
Please change order of modules in mentioned Path(s) to avoid inconsistent module ordering.
----- End Fatal Exception -------------------------------------------------

tocheng · 2018-10-26T05:08:27Z

@fabiocos
@mmusich did some test. The error has nothing to do with PR and doesn't appear in 10_3_X

Hello @fabiocos, @slava77,
while this PR runs fine in the last IB of 10.3.X using:

python $CMSSW_BASE/src/Configuration/DataProcessing/test/RunExpressProcessing.py --scenario ppEra_Run2_2018_pp_on_AA --global-tag 103X_dataRun2_Express_v1 --lfn /store/t0streamer/Data/HIExpress/000/325/112/run325112_ls0013_streamHIExpress_StorageManager.dat --fevt --dqmio --alcareco DtCalib

and then changing the event source with:

process.source = cms.Source("NewEventStreamFileReader"

while doing the same in 10.4.X I notice I incur in this error:

----- Begin Fatal Exception 25-Oct-2018 22:56:57 CEST-----------------------
An exception of category 'ScheduleExecutionFailure' occurred while
   [0] Calling beginJob
Exception Message:
Unrunnable schedule
Module run order problem found: 
hiConformalPixelTracks after hiConformalPixelTracksPhase1Filter [path reconstruction_step], hiConformalPixelTracksPhase1Filter after pixelFitterByConformalMappingAndLine [path reconstruction_step], pixelFitterByConformalMappingAndLine after hiConformalPixelTracksPhase1HitQuadrupletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitQuadrupletsCA after hiConformalPixelTracksPhase1HitDoubletsCA [path reconstruction_step], hiConformalPixelTracksPhase1HitDoubletsCA after hiConformalPixelTracksPhase1SeedLayers [path reconstruction_step], hiConformalPixelTracksPhase1SeedLayers after hiConformalPixelTracksPhase1TrackingRegions [path reconstruction_step], hiConformalPixelTracksPhase1TrackingRegions after hiClusterCompatibility [path reconstruction_step], hiClusterCompatibility after hiCentrality [path reconstruction_step], hiCentrality consumes hiConformalPixelTracks
 Running in the threaded framework would lead to indeterminate results.
 Please change order of modules in mentioned Path(s) to avoid inconsistent module ordering.
----- End Fatal Exception -------------------------------------------------

the same error is also obtained in a vanilla CMSSW_10_4_X_2018-10-25-1100.
Is there any substantial difference between the tip of 10.3.X and the tip of 10.4.X that can cause this?

fabiocos · 2018-10-26T07:06:51Z

ok, I confirm that with the latest fix of @slava77 #25011 the PR looks to work

fabiocos · 2018-10-26T07:07:00Z

+1

Using pp_on_AA_2018 era to configure DtCalib AlCaReco the same as DtC…

29bc4d6

…alibHI

cmsbuild added this to the CMSSW_10_4_X milestone Oct 25, 2018

cmsbuild added alca-pending code-checks-pending comparison-pending orp-pending pending-signatures tests-pending labels Oct 25, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 25, 2018

cmsbuild added tests-started and removed tests-pending labels Oct 25, 2018

mmusich reviewed Oct 25, 2018

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Oct 25, 2018

Add offlinePrimaryVertices instead of hiSelectedVertex

dfc339c

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved tests-approved labels Oct 25, 2018

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 25, 2018

cmsbuild added tests-started and removed tests-pending labels Oct 25, 2018

tocheng mentioned this pull request Oct 25, 2018

Configure DtCalib AlCaReco for heavy Ion 2018 #25008

Merged

cmsbuild added tests-approved and removed tests-started labels Oct 26, 2018

cmsbuild added comparison-available and removed comparison-pending labels Oct 26, 2018

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Oct 26, 2018

This was referenced Oct 26, 2018

Merge of 24892 crashes workflow 148.0 #24981

Closed

streamline addition of centrality and pixel tracks for HI #25011

Merged

cmsbuild added orp-approved and removed orp-pending labels Oct 26, 2018

cmsbuild merged commit 29a4912 into cms-sw:master Oct 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Configure DtCalib AlCaReco for heavy Ion 2018 #25004

Configure DtCalib AlCaReco for heavy Ion 2018 #25004

tocheng commented Oct 25, 2018 •

edited

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

tocheng commented Oct 25, 2018

cmsbuild commented Oct 25, 2018 •

edited

mmusich Oct 25, 2018

tocheng Oct 25, 2018

mmusich commented Oct 25, 2018 •

edited

mandrenguyen commented Oct 25, 2018 via email

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

tocheng commented Oct 25, 2018

cmsbuild commented Oct 25, 2018 •

edited

cmsbuild commented Oct 26, 2018

cmsbuild commented Oct 26, 2018

cmsbuild commented Oct 26, 2018

tocheng commented Oct 26, 2018

cmsbuild commented Oct 26, 2018

fabiocos commented Oct 26, 2018

tocheng commented Oct 26, 2018 •

edited

fabiocos commented Oct 26, 2018

fabiocos commented Oct 26, 2018

Configure DtCalib AlCaReco for heavy Ion 2018 #25004

Configure DtCalib AlCaReco for heavy Ion 2018 #25004

Conversation

tocheng commented Oct 25, 2018 • edited

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

tocheng commented Oct 25, 2018

cmsbuild commented Oct 25, 2018 • edited

mmusich Oct 25, 2018

Choose a reason for hiding this comment

tocheng Oct 25, 2018

Choose a reason for hiding this comment

mmusich commented Oct 25, 2018 • edited

mandrenguyen commented Oct 25, 2018 via email

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

cmsbuild commented Oct 25, 2018

tocheng commented Oct 25, 2018

cmsbuild commented Oct 25, 2018 • edited

cmsbuild commented Oct 26, 2018

cmsbuild commented Oct 26, 2018

cmsbuild commented Oct 26, 2018

tocheng commented Oct 26, 2018

cmsbuild commented Oct 26, 2018

fabiocos commented Oct 26, 2018

tocheng commented Oct 26, 2018 • edited

fabiocos commented Oct 26, 2018

fabiocos commented Oct 26, 2018

tocheng commented Oct 25, 2018 •

edited

cmsbuild commented Oct 25, 2018 •

edited

mmusich commented Oct 25, 2018 •

edited

cmsbuild commented Oct 25, 2018 •

edited

tocheng commented Oct 26, 2018 •

edited