[90X] Allow multi-IOV input for Millepede alignment framework #17475

ghellwig · 2017-02-09T15:29:26Z

Removes shortcoming of the alignment framework which has not been able to update the alignables after an update of the tracker geometry at an IOV boundary.

Removes also temporary fix of #16135 and enables the usage of multi-IOV input for Millepede.

Added various protection against misconfiguration.

consistency check of the configuration during the setup of the alignment campaign
consistency checks in the C++ code to detect misconfiguration

The code has been used to derive the 2016 legacy alignment.

The general AlignmentProducer has the new features disabled by default. It is only enabled for Millepede. If other alignment algorithms want to use it a flag has to be set to True:

process.AlignmentProducer.enableAlignableUpdates = True

The code is based on top of #17408, i.e. to better digest the changes, it might be useful to first review and eventually approve #17408.

EDIT: A dedicated presentation on this update has been given at the Tracker Alignment meeting on 01-Mar-2017

When requesting information of all 2016 MinBias datasets, the runtime is reduced from 2-3 hours to approx. 2 minutes.

Changes commmited on behalf of Rainer Mankel. Details are presented here: https://indico.cern.ch/event/607492/#35-update-on-upsilonmumu-const

- move subtask into dedicated methods -> first step needed for unification -> also needed for support of multi-IOV input tag

ghellwig · 2017-03-07T11:43:09Z

please test

cmsbuild · 2017-03-07T11:43:25Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/18202/console Started: 2017/03/07 12:45

cmsbuild · 2017-03-07T13:28:35Z

+1
Tested at: ffa383d
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17475/18202/summary.html

cmsbuild · 2017-03-07T13:28:37Z

Comparison job queued.

franzoni · 2017-03-07T14:56:02Z

Alignment/MillePedeAlignmentAlgorithm/python/mpslib/Mpslibclass.py

+            self.JOBREMARK.append(parts[9].strip())
+            self.JOBSP1.append(parts[10].strip())
+            self.JOBSP2.append(parts[11].strip())
+            self.JOBSP3.append(parts[12].strip())


hello @ghellwig
how do you know which elements of parts[]
you're in need of removing blanks from ?

Everything that is not converting to an integer remains a string and can have unintended whitespace at the left and right end.
The conversion to integers ignores such whitespace anyways.

cmsbuild · 2017-03-07T15:30:46Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-17475/18202/summary.html

franzoni · 2017-03-07T15:38:31Z

no changes in plots 2017 ttbar, 2016B data, runII2016 25 and 50 ns

franzoni · 2017-03-07T15:38:35Z

+1

cmsbuild · 2017-03-07T15:39:24Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_9_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

franzoni · 2017-03-14T15:06:13Z

including this PR 900 important for alignment
may we have this PR merged, please ?

davidlange6 · 2017-03-14T17:18:36Z

+1

gregor-mittag added 30 commits February 3, 2017 11:40

Fix 'mps_parse_pedechi2hist.py'.

348a248

Fix variable replacement in pede-script-template.

cfa7827

Fix style of 'mps_check.py'.

37a129b

Add handling of missing output file (rare case).

3c9729d

Use context manager to open files in 'mps_check.py'.

c6779db

Add protection against accidentally using data prior to first IOV.

079762c

Ignore auto-generated ROOT file in MPS validation.

e1ef6b6

Fix style of 'mps_fetch.py'.

5627d8c

Add recovery from previous failures in 'mps_fetch.py'.

b81a6b9

Fixed memory usage calculation.

06cc2da

Fix input validation in 'mps_create_file_lists.py'.

a83aac4

Fix automatic single-IOv input for '-w' option of 'mps_alisetup.py'.

c12007b

Further improve input validation of mps_create_file_lists.py.

8326655

Make mps_create_file_lists.py more robust against DAS issues.

7aeb91b

Fixed shuffling of files in case a DAS cache is used.

a0feb84

Create dataset cff-files also for alignment datasets.

9c6aa5b

Enable forwarding of proxy for MPS.

3dfacc6

Fix '-f/--force-merge' option of 'mps_fire.py'.

d53ffda

Fix behaviour of mps_create_file_lists.py if only help is wanted.

b7dce9f

Speed-up of file creation script.

d63f96a

When requesting information of all 2016 MinBias datasets, the runtime is reduced from 2-3 hours to approx. 2 minutes.

Better log message for file-list creation script.

bc6e64d

Enable usage of UpsilonMuMu data in MillePede workflow.

bc73c80

Changes commmited on behalf of Rainer Mankel. Details are presented here: https://indico.cern.ch/event/607492/#35-update-on-upsilonmumu-const

Define default pede settings and provide helper to customize them.

3815b8d

Partly align AlignmentProducer to PCLTrackerAlProducer.

209a96e

- move subtask into dedicated methods -> first step needed for unification -> also needed for support of multi-IOV input tag

Add check if setup has changed to 'AlignmentProducer'.

f53f30e

Move common function to utilities.

0e8bafc

Added 'update' method to 'Alignable'.

8728cd9

Added 'update' method to 'AlignableDetUnit'.

5697a09

Added 'reset' method to 'AlignableBeamSpot'.

eee77d2

Added 'update' methods to 'AlignableComposite'.

9983c84

cmsbuild added comparison-pending tests-pending tests-started and removed comparison-available tests-approved tests-pending labels Mar 7, 2017

cmsbuild added tests-approved and removed tests-started labels Mar 7, 2017

franzoni reviewed Mar 7, 2017

View reviewed changes

cmsbuild added comparison-available and removed comparison-pending labels Mar 7, 2017

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Mar 7, 2017

cmsbuild added orp-approved and removed orp-pending labels Mar 14, 2017

cmsbuild merged commit 937a0f7 into cms-sw:CMSSW_9_0_X Mar 14, 2017

davidlange6 mentioned this pull request Mar 24, 2017

Fix bug in L1tMuonProducer #18063

Closed

ghellwig deleted the allow_multi-iov_millepede_90X branch May 2, 2017 15:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[90X] Allow multi-IOV input for Millepede alignment framework #17475

[90X] Allow multi-IOV input for Millepede alignment framework #17475

ghellwig commented Feb 9, 2017 •

edited

ghellwig commented Mar 7, 2017

cmsbuild commented Mar 7, 2017 •

edited

cmsbuild commented Mar 7, 2017

cmsbuild commented Mar 7, 2017

franzoni Mar 7, 2017

ghellwig Mar 7, 2017

cmsbuild commented Mar 7, 2017

franzoni commented Mar 7, 2017

franzoni commented Mar 7, 2017

cmsbuild commented Mar 7, 2017

franzoni commented Mar 14, 2017

davidlange6 commented Mar 14, 2017

[90X] Allow multi-IOV input for Millepede alignment framework #17475

[90X] Allow multi-IOV input for Millepede alignment framework #17475

Conversation

ghellwig commented Feb 9, 2017 • edited

ghellwig commented Mar 7, 2017

cmsbuild commented Mar 7, 2017 • edited

cmsbuild commented Mar 7, 2017

cmsbuild commented Mar 7, 2017

franzoni Mar 7, 2017

Choose a reason for hiding this comment

ghellwig Mar 7, 2017

Choose a reason for hiding this comment

cmsbuild commented Mar 7, 2017

franzoni commented Mar 7, 2017

franzoni commented Mar 7, 2017

cmsbuild commented Mar 7, 2017

franzoni commented Mar 14, 2017

davidlange6 commented Mar 14, 2017

ghellwig commented Feb 9, 2017 •

edited

cmsbuild commented Mar 7, 2017 •

edited