Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[90X] Allow multi-IOV input for Millepede alignment framework #17475

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
348a248
Fix 'mps_parse_pedechi2hist.py'.
gregor-mittag Oct 10, 2016
cfa7827
Fix variable replacement in pede-script-template.
gregor-mittag Oct 11, 2016
37a129b
Fix style of 'mps_check.py'.
gregor-mittag Oct 11, 2016
3c9729d
Add handling of missing output file (rare case).
gregor-mittag Oct 11, 2016
c6779db
Use context manager to open files in 'mps_check.py'.
gregor-mittag Oct 11, 2016
079762c
Add protection against accidentally using data prior to first IOV.
gregor-mittag Oct 11, 2016
e1ef6b6
Ignore auto-generated ROOT file in MPS validation.
gregor-mittag Oct 11, 2016
5627d8c
Fix style of 'mps_fetch.py'.
gregor-mittag Oct 13, 2016
b81a6b9
Add recovery from previous failures in 'mps_fetch.py'.
gregor-mittag Oct 13, 2016
06cc2da
Fixed memory usage calculation.
gregor-mittag Oct 14, 2016
a83aac4
Fix input validation in 'mps_create_file_lists.py'.
gregor-mittag Oct 14, 2016
c12007b
Fix automatic single-IOv input for '-w' option of 'mps_alisetup.py'.
gregor-mittag Nov 4, 2016
8326655
Further improve input validation of mps_create_file_lists.py.
gregor-mittag Nov 18, 2016
7aeb91b
Make mps_create_file_lists.py more robust against DAS issues.
gregor-mittag Nov 18, 2016
a0feb84
Fixed shuffling of files in case a DAS cache is used.
gregor-mittag Nov 21, 2016
9c6aa5b
Create dataset cff-files also for alignment datasets.
gregor-mittag Nov 24, 2016
3dfacc6
Enable forwarding of proxy for MPS.
gregor-mittag Nov 24, 2016
d53ffda
Fix '-f/--force-merge' option of 'mps_fire.py'.
gregor-mittag Nov 29, 2016
b7dce9f
Fix behaviour of mps_create_file_lists.py if only help is wanted.
gregor-mittag Dec 22, 2016
d63f96a
Speed-up of file creation script.
gregor-mittag Dec 22, 2016
bc6e64d
Better log message for file-list creation script.
gregor-mittag Jan 31, 2017
bc73c80
Enable usage of UpsilonMuMu data in MillePede workflow.
gregor-mittag Jan 31, 2017
3815b8d
Define default pede settings and provide helper to customize them.
gregor-mittag Jan 31, 2017
209a96e
Partly align AlignmentProducer to PCLTrackerAlProducer.
gregor-mittag Oct 13, 2016
f53f30e
Add check if setup has changed to 'AlignmentProducer'.
gregor-mittag Oct 13, 2016
0e8bafc
Move common function to utilities.
gregor-mittag Nov 14, 2016
8728cd9
Added 'update' method to 'Alignable'.
gregor-mittag Nov 14, 2016
5697a09
Added 'update' method to 'AlignableDetUnit'.
gregor-mittag Nov 14, 2016
eee77d2
Added 'reset' method to 'AlignableBeamSpot'.
gregor-mittag Nov 14, 2016
9983c84
Added 'update' methods to 'AlignableComposite'.
gregor-mittag Nov 14, 2016
09feebb
Added 'update' method to 'AlignableDet'.
gregor-mittag Nov 14, 2016
9a5ec4a
Added 'update' methods to muon AlignableDets.
gregor-mittag Nov 14, 2016
65e8e5d
Added beam-spot resetting function to 'AlignableExtras'.
gregor-mittag Nov 14, 2016
08f8718
Support updates in AlignableTrackerBuilder and AlignableCompositeBuil…
gregor-mittag Nov 18, 2016
0e649cc
Added 'update' methods to muon 'AlignableTracker' and 'AlignableMuon'.
gregor-mittag Nov 18, 2016
d5351cd
Update alignables at IOV boundaries in AligmmentProducer.
gregor-mittag Nov 14, 2016
4c3d46a
Add run-based caching to Alignables.
gregor-mittag Nov 14, 2016
88bdf8b
Add option to skip check for GlobalPositionRcd IOV boundary.
gregor-mittag Nov 14, 2016
b8ac129
Removed temporary protection against multi-IOV input.
gregor-mittag Nov 14, 2016
9cd0110
Added SurfaceDeformations to 'AlignableDetUnit::dump()'.
gregor-mittag Nov 11, 2016
b58ff33
Added check for pede step, if all needed runs have been processed.
gregor-mittag Nov 14, 2016
2066fd5
Added flag to enable/disable updatable alignables.
gregor-mittag Nov 14, 2016
fd45aec
Allow multiple levels of IOVs.
gregor-mittag Dec 12, 2016
ce8ef68
Adapt single-IOV creation script to sqlalchemy 1.1.4
gregor-mittag Feb 7, 2017
8e7d7a6
Move common function to mps tools.
gregor-mittag Feb 7, 2017
c5071a2
Separate process extraction from input-db creation.
gregor-mittag Feb 7, 2017
56f6550
Implemented early detection of inconsistent IOV definitions in MPS.
gregor-mittag Feb 9, 2017
fb83c79
Added unit test for database-query script.
gregor-mittag Feb 9, 2017
150d6f6
Allow multi-IOV input in pixel-barycenter tool.
gregor-mittag Feb 9, 2017
831c2e9
Remove redundant headers.
gregor-mittag Feb 9, 2017
edfcd64
Adjust PCL alignment producer to multi-IOV alignment framework.
gregor-mittag Feb 10, 2017
ffa383d
Fix clang warnings caused by 'update' method.
gregor-mittag Feb 14, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -99,18 +99,18 @@ def read_db(self, db_file_name = __default_db_file_name):
line = line.rstrip('\n') #removes the pesky \n from line
parts = line.split(":") #read each line and split into parts list
self.JOBNUMBER.append(int(parts[0]))
self.JOBDIR.append(parts[1])
self.JOBDIR.append(parts[1].strip())
self.JOBID.append(int(parts[2]))
self.JOBSTATUS.append(parts[3])
self.JOBSTATUS.append(parts[3].strip())
self.JOBNTRY.append(int(parts[4]))
self.JOBRUNTIME.append(int(parts[5])) #int float?
self.JOBNEVT.append(int(parts[6]))
self.JOBHOST.append(parts[7])
self.JOBHOST.append(parts[7].strip())
self.JOBINCR.append(int(parts[8]))
self.JOBREMARK.append(parts[9])
self.JOBSP1.append(parts[10])
self.JOBSP2.append(parts[11])
self.JOBSP3.append(parts[12])
self.JOBREMARK.append(parts[9].strip())
self.JOBSP1.append(parts[10].strip())
self.JOBSP2.append(parts[11].strip())
self.JOBSP3.append(parts[12].strip())
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hello @ghellwig
how do you know which elements of parts[]
you're in need of removing blanks from ?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything that is not converting to an integer remains a string and can have unintended whitespace at the left and right end.
The conversion to integers ignores such whitespace anyways.


#count number of jobs
if not self.JOBDIR[self.nJobs].startswith("jobm"):
Expand Down
3 changes: 2 additions & 1 deletion Alignment/MillePedeAlignmentAlgorithm/scripts/mps_fetch.py
Expand Up @@ -19,7 +19,8 @@

# loop over DONE jobs
for i in xrange(len(lib.JOBID)):
if 'DONE' in lib.JOBSTATUS[i]:
# check also "FETCH" to recover from possibly failed runs of 'mps_fetch.py'
if lib.JOBSTATUS[i] in ("DONE", "FETCH", "DISABLEDFETCH"):
# move the LSF output to /jobData/
theJobDir = 'jobData/'+lib.JOBDIR[i]
theBatchDirectory = 'LSFJOB\_%d' % lib.JOBID[i]
Expand Down