New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
assure no DS collision LHE relvals #6692
assure no DS collision LHE relvals #6692
Conversation
franzoni
commented
Nov 28, 2014
- needs to go on top of Adding first LHE sequence to RelVal #6626 (already rebased on top of CMSSW_7_4_X)
- enforces that no LHE workflow can produce the same /GEN dataset
- /GEN recycling command all point to the same /GEN dataset
…E_13TeV' will recycle the same dataset, from wf [512] of generator set
A new Pull Request was created by @franzoni (Giovanni Franzoni) for CMSSW_7_4_X. assure no DS collision LHE relvals It involves the following packages: Configuration/PyReleaseValidation @cmsbuild, @srimanob, @nclopezo, @boudoul, @franzoni can you please review it and eventually sign? Thanks. |
+1 |
This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs unless changes or unless it breaks tests. |
This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs unless changes (tests are also fine). This pull request will be automatically merged. |
…W_7_4_X assure no DS collision LHE relvals
Seems that these workflows are failing in the IBs - is that expected? |
hello David, the crashes for the 6 new relvals happen because of the input dataset recycle used by default in the IB: until when we'll be able to run the workflows for the first time, the input won't be found. Concerning cvmfs : if we remove the non existing input dataset, cvmfs will become necessary for the workflow to function. It is my understanding that cvmfs is installed throughout our computing infrastructure, GEN workflows exploit that assumption. If you know for a fact that cmvf is not available on the IB machines (though are you sure ? Alessandro actually helped me to install it on my machine : https://twiki.cern.ch/twiki/bin/viewauth/CMS/CmsIbCvmfs), Cheers, |
On the general policy: we will try to avoid the cvmfs dependency for most GEN-SIM workflows, but for LHE workflows it is unavoidable, since the use of "alternate pdf weights" implies a large set of pdfs are used by the workflows, and they cannot all be included directly in the release. |
On the error message, this is tricky because it is not possible for cmsDriver or even the ExternalLHEProducer to definitively know a priori whether the particular workflow really requires cvmfs. |
Hi Josh, the error message is simple to digest: Pythia at the step2 is trying to open this file : 29-Nov-2014 15:13:20 CET Closed file root://eoscms//eos/cms/store/data/Run2012B/MinimumBias/RAW/v1/000/194/533/0C1BB1C7-ACA2-E111-A831-003048D3750A.root?svcClass=default :-P |
@franzoni, sorry I was referring to David's comment about the error message when cvmfs is missing. |
Regarding cvmfs- my question is not for the IB infrastructure but for users. On Nov 29, 2014, at 7:13 PM, Giovanni Franzoni <notifications@github.commailto:notifications@github.com> wrote: hello David, the crashes for the 6 new relvals happen because of the input dataset recycle used by default in the IB: until when we'll be able to run the workflows for the first time, the input won't be found. Concerning cvmfs : if we remove the non existing input dataset, cvmfs will become necessary for the workflow to function. It is my understanding that cvmfs is installed throughout our computing infrastructure, GEN workflows exploit that assumption. If you know for a fact that cmvf is not available on the IB machines (though are you sure ? Alessandro actually helped me to install it on my machine : https://twiki.cern.ch/twiki/bin/viewauth/CMS/CmsIbCvmfs), Cheers, — |
Hi Josh Remind me how big these particular files are? (The ones used by relvals which I presume will now be essentially fixed) On Nov 29, 2014, at 7:17 PM, Josh Bendavid <notifications@github.commailto:notifications@github.com> wrote: On the general policy: we will try to avoid the cvmfs dependency for most GEN-SIM workflows, but for LHE workflows it is unavoidable, since the use of "alternate pdf weights" implies a large set of pdfs are used by the workflows, and they cannot all be included directly in the release. — |
Well, also in the IB infrastructure, while CVMFS might be available on some machines, we do not guarantee (e.g. on mac / arm) it is actually there. |