Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DQM Physics sequence missing input collections #22996

Closed
fabiocos opened this issue Apr 18, 2018 · 27 comments
Closed

DQM Physics sequence missing input collections #22996

fabiocos opened this issue Apr 18, 2018 · 27 comments

Comments

@fabiocos
Copy link
Contributor

The DQM Physics sequence is throwing exceptions in the express configuration due to the lack of an input jet corrector collection. A quick protection has been made available in #22992 / #22993 to prevent on the very short term express crashes even if the collection is missing, but this is just a temporary workaround.

A proper solution implies fixing the DQM Physics sequence, in a similar way as the DQM JetMET sequence, so as to pass correctly the collection.

Two solution have been checked to solve the problem in the express configuration:

  • adding the ak4PFCHSL1FastL2L3CorrectorChain sequence at the beginning of the dqmPhysics sequence;

  • move the name of the collection to the clone dqmAk4PFCHSL1FastL2L3CorrectorChain, used by the DQM JetMET sequence, in the top physics modules requiring it.

A proper solution should be deplyed, making the protection in #22992 not necessary, but it should be verified to work in all possible scenarios.

@cmsbuild
Copy link
Contributor

A new Issue was created by @fabiocos Fabio Cossutti.

@davidlange6, @Dr15Jones, @smuzaffar, @fabiocos can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@fabiocos
Copy link
Contributor Author

assign dqm

@cmsbuild
Copy link
Contributor

New categories assigned: dqm

@jfernan2,@vazzolini,@vanbesien,@kmaeshima,@dmitrijus you have been requested to review this Pull request/Issue and eventually sign? Thanks

@fabiocos
Copy link
Contributor Author

@jfernan2 is your revision of DQM sequences thought to address also this issue?

@jfernan2
Copy link
Contributor

@fabiocos it was not but I am adding it now to the list
Are 1001.2,1040.1 workflows still good to test this?
Thanks

@jfernan2
Copy link
Contributor

jfernan2 commented Oct 15, 2019

BTW: I am not sure why 'assign dqm' is retrieving an old list of DQM L2 people since cms-bot categories.py file has the right list....
Where can I fix that? Sorry I failed to find it out by myself on github...

@davidlange6
Copy link
Contributor

davidlange6 commented Oct 15, 2019 via email

@jfernan2
Copy link
Contributor

@davidlange6 as I said in my comment, the categories.py file is up to date in what dqm concerns since a long time, just search for @vazzolini,@vanbesien,@dmitrijus there, they are not there but the comment by cmsbuild triggered by "assign dqm" command took a wrong list

@davidlange6
Copy link
Contributor

davidlange6 commented Oct 15, 2019 via email

@fabiocos
Copy link
Contributor Author

@jfernan2 @davidlange6 please note that the issue itself is quite old, open since April last year...

@fabiocos
Copy link
Contributor Author

As now @jfernan2 is going through the sequence cleaning process, I think it is a good idea to check back all the issues that were temporarily hidden during data taking by protections

@jfernan2
Copy link
Contributor

@jfernan2 @davidlange6 please note that the issue itself is quite old, open since April last year...

you are right... my fault then

@jfernan2
Copy link
Contributor

@fabiocos the point is how to check at present if the fix works, 1001.2,1040.1 workflows do not show this problem with runTheMatrix.py as far as I can tell from the logs..

@fabiocos
Copy link
Contributor Author

@jfernan2 this goes back to the problem of defining a proper express-like test suitable for recent runs. Configuration/DataProcessing can provide a tool to help here.

jfernan2 added a commit to jfernan2/cmssw that referenced this issue Oct 16, 2019
@jfernan2
Copy link
Contributor

@fabiocos I have fixed the issue in the last commit of #28156
I used the second solution proposed in the issue since it seems more CPU economic...
I have tested it offline and works, but I am afraid it will change again for Express Run3 if Puppy Jets are finally adopted
On the other side don't really know how to introduce a test for this/Express in github/Jenkins...

@fabiocos
Copy link
Contributor Author

@jfernan2 thank you, defining a test workflow to mimic the latest express configuration is something we should consider, as discussed multiple times

@jfernan2
Copy link
Contributor

This issue should be closed

@fabiocos
Copy link
Contributor Author

@jfernan2 @silviodonato do we have evidence that all the issues (i.e. error messages induced by protections) have gone? And was the express workflow tested in some way?

@silviodonato
Copy link
Contributor

Good point @fabiocos . I tried to look for the error messages related to the protection in the IB tests of old releases without success. @jfernan2, do you know how to reproduce the old error related to topSingleMuonMediumDQM, topSingleElectronMediumDQM, singleTopMuonMediumDQM, singleTopElectronMediumDQM?

@jfernan2
Copy link
Contributor

@silviodonato I guess that reverting the PR i made up there, right?

@jfernan2
Copy link
Contributor

Sorry @silviodonato I do not understand: the link you point out is from IB of today while #28156 was merged in 11_0_X around October 2019
I am sorry but I do not follow what you mean

@jfernan2
Copy link
Contributor

I see... @fabiocos spotted this in real Express sequence at Tier0 if I understand correctly. 136.8391 is using DQM:@standardDQM+@ExtraHLT+@miniAODDQM so the issue should be hidden there since all the JetCorrections are loaded

@schneiml has been trying to get the actual config file and cmsRun environment at T0:
https://its.cern.ch/jira/browse/CMSTZ-427
But no success yet.

Nevertheless, I have tested DQM/Physics/test/topDQM_production_cfg.py which is the closest test file to the TOP modules, and since the Jet correctors are loaded in the JetMET DQM module the problem should be gone now even in real express...

@fabiocos
Copy link
Contributor Author

@jfernan2 @silviodonato @qliphy not sure which kind of updates are available nowadays, some cleaning was ongoing already when I left the release manager task. Probably this issue should be closed and something newer opened in case there are residual problems are present, please provide comments and suggestions.

@silviodonato
Copy link
Contributor

I agree, I think we can close this issue. @jfernan2 ?

@jfernan2
Copy link
Contributor

Yes, fine from my side

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants