Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[90X] Update of the Millepede Production System #17408

Merged
merged 23 commits into from Feb 17, 2017

Conversation

ghellwig
Copy link

@ghellwig ghellwig commented Feb 3, 2017

Summary of changes

  • speed improvement in dataset creation script
  • proxy forwarding for batch jobs (needed for xrootd access)
  • helper function to ease the configuration of pede
  • various smaller fixes
  • support for UpsilonMuMu datasets

When requesting information of all 2016 MinBias datasets, the runtime is reduced
from 2-3 hours to approx. 2 minutes.
@ghellwig
Copy link
Author

ghellwig commented Feb 3, 2017

please test

@cmsbuild cmsbuild added this to the Next CMSSW_9_0_X milestone Feb 3, 2017
@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 3, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17611/console Started: 2017/02/03 13:42

@ghellwig
Copy link
Author

ghellwig commented Feb 6, 2017

please test
failure seems unrelated

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 6, 2017

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/17624/console Started: 2017/02/06 10:16

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 6, 2017

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 6, 2017

Comparison job queued.

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 6, 2017

<< "@SUB=MillePedeAlignmentAlgorithm::beginRun\n"
<< "Using data (run = " << run.run()
<< ") prior to the first defined IOV (" << firstIOV_ << ").";
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @ghellwig
for information, when the framework was to look for conditions not covered by any IOV, this : https://hypernews.cern.ch/HyperNews/CMS/get/dataopsrequests/17299/1.html is what happens

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @franzoni,
yes, I have seen this failure, but this check has a different purpose.
This is checking the output IOV definition, e.g. if the alignment is supposed to deliver IOVs starting from run 200000 and data is used from , e.g., run 190000.
Without the check, the resulting alignment of the high-level structures would not be affected because the data does not fall into any IOV, but the module-level alignment would be slightly affected because the modules are assumed to be constant wrt. the high-level structures, i.e. all data is used for the module-level alignment. That means data from all IOVs and without this check even from periods prior to any defined output IOV.
In addition this check also prevents problems in a second iteration where the input IOVs are the output IOVs from the first iteration. In this case, we would get the kind of message you are pointing to, but only after running an alignment which might have taken up to two days.

@franzoni
Copy link

+1

impressive set of improvements, on several technical fronts as well as expansion of measurement potential (ALCARECOTkAlUpsilonMuMu). No changes to 2017, 2016 data/mc central tests.

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_9_0_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @smuzaffar

@davidlange6
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit d238cf4 into cms-sw:CMSSW_9_0_X Feb 17, 2017
@ghellwig ghellwig deleted the MPS_updates_90X branch February 17, 2017 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants