Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update master-102X to CMSSW_10_2_9 #279

Closed
wants to merge 152 commits into from
Closed

Conversation

peruzzim
Copy link

Includes #230 #238 #254 #267 #273

henriettepetersen and others added 30 commits September 28, 2018 19:15
If no CUDA devices are present, skip the tests and return a success
status, instead of failing.
Skip tests if no CUDA devices are present (10.2.x)
…acked-parameters-10_2_X

backport of EM enrichment filters parameters tracking PR cms-sw#24754
peruzzim and others added 10 commits December 11, 2018 14:56
Conflicts:
	PhysicsTools/NanoAOD/python/nano_cff.py
	PhysicsTools/NanoAOD/python/photons_cff.py
[102X] Change GT. Offline GT Fix ECAL Tags
Backport updated ecalBadCalibFilter crystal list [102X]
…cer_10_2_X

[10_2_X] Fall17 V2 Photon ID and PhotonIDValueMapProducer speedup
* First implementation of deep tau id.

* Building dpf isolation module

* Adding in v1

* Adding in runTauIDMVA for other users

* making things fully reproducible

* Reorganisation of configuration files: cff split to cfi and cff

* Some code cleaning

* adapt to cfi/cff reorganization

* Review of DPF and DeepTauId code.

- Defined base class for deep tau discriminators.
- Removed weight files from home cms repository. Now using weights from cms-data.
- Defined WP for both discriminators. Now all discriminators return the corresponding WP results.
- Removed cfi files. Using fillDescriptions instead.
- General code review and cleaning.

* Added example of a python configuration file to produce pat::Tau collection with the new Tau-Ids

* requested changes on runDeepTauIDsOnMiniAOD.py

* Clean runTauIdMVA.py tool and test config to run tauIDs

* Made DeepTauId and DPFIsolation thread-safe

* Finish implement thread-safe requirements on DPFIsolation

* Disable DPFTau_2016_v1 and issue some warnings

* Remove assigning value of variable to itself

* - Implemented on runTauIdMVA the option to work with new training files quantized
- Added a new parameter 'version' on runTauIdMVA, used on DPFIsolation
- Changes on DeepTauId to reduce memory consumption

* - Implementation of global cache to avoid reloading graph for each thread and reduce the memory consuption
- Creation of class DeepTauCache in DeepTauBase, in which now is created graph and session
- Implementation of two new static methods inside the class DeepTauBase: initializeGlobalCache and globalEndJob. The graph and DeepTauCache object are created now inside initializeGlobalCache

* Applied changes on DeepTauBase to allow load new training files using memory mapping

* Implemented TauWPThreshold class.

TauWPThreshold class parses WP cut string (or value) provided in the
python configuration. It is needed because the use of the standard
StringObjectFunction class to parse complex expression results in an
extensive memory usage (> 100 MB per expression).

* Remove the qm.pb input files and leaving just the quantized and the original files

* -Overall, changes to improve memory usage, among these are:
	- Implementation of global cache to avoid reloading graph for each thread
	- Creation of two new static methods inside the class DeepTauBase: initializeGlobalCache and globalEndJob. The graph and DeepTauCache object are created now inside initializeGlobalCache. The memory consumption of initializeGlobalCache for the original, quantized and files that are load using memory mapping method are in the memory_usage.pdf file
	- Implemented configuration to use new training files quantized, and set them as default
	- Implementation of configuration for load files using memory mapping. In our case there wasn't any improvement, respect at the memory consumption of this method, respect the quantized files, so this is not used, but set for future training files
- General code review and cleaning.

* Applied style comments

* Applied style comments

* Applied comments

* Change to be by default the original training file for deepTau, instead of the quantized

* Changes regarding forward-porting DNN-related developments from the PRs #105 and #106 from 94X to 104X

* Applied commets of previus PR

* cleaning code

* Modification in the config to work with new label in files

* Applied comment about the expected format of name of training file

* Fix in last commit

* Applied last comments

* Changes regarding forward-porting DNN-related developments from the PRs #105 and #106 from 94X to 104X

* Applied @perrotta comments on 104X

* Fix error

* Applied comments

* Applied comments

* Fix merge problem

* Applied a few commets

* Applied more changes

* Applied a few small followups

*  Fixed error on DPFIsolation

* Update DPFIsolation.cc

* - RecoTauTag/RecoTau/plugins/DeepTauId.cc: Remove ' clusterVariables 'as a  class member
- RecoTauTag/RecoTau/test/runDeepTauIDsOnMiniAOD.py: Update globaltag and sample

* Added changes in RecoTauTag/RecoTau/python/tools/runTauIdMVA.py made in the commit 194a1d5 from the PR cms-sw#25016

* Fix error on runDeepTauIDsOnMiniAOD

* Change the GT in RecoTauTag/RecoTau/test/runDeepTauIDsOnMiniAOD.py
NanoAOD developments, 29Nov2018 version [102X]
Add to IB/release history PR 25386 DNN-based Tau-Id discrimians (102X)
@gpetruc-bot
Copy link

Copy link

@gpetruc-bot gpetruc-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automatic test report for 634962

Code integration

Code checks passed for this PR

Tests

  • Test mc_94Xv2: passed
  • Test mc_102X: passed
  • Test data_94X: passed
  • Test data_101X: passed

@gpetruc-bot
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet