Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make DetSetNew to support concurrent filler #12411

Merged
merged 34 commits into from Nov 23, 2015

Conversation

VinInn
Copy link
Contributor

@VinInn VinInn commented Nov 13, 2015

With this PR DetSetNew support concurrent fillers.
The main application is Strip Unpacking and Clusterization on Demand at HLT.

Passed all tests and show no regression in sequential mode.

How to test the concurrency (besides the unit test)?
Waiting for the in-the-event module concurrency you can try concurrent PatternReco

git cms-addpkg RecoTracker/CkfPattern
touch RecoTracker/CkfPattern/src/CkfTrackCandidateMakerBase.cc
scram b -j8 USER_CXXFLAGS="-DVI_TBB"

take 134.91
add

process.options.numberOfThreads=cms.untracked.uint32(8)
process.options.numberOfStreams=cms.untracked.uint32(1)

to step2_L1REPACK_HLT.py
and

cmsRun step2_L1REPACK_HLT.py 

you can also switch VI_DEBUG in ClustersFromRawProducer.cc and compare
the dump (better just the event by event VI VI clusters lines)

At the moment it will show some differences due to the fact that concurrent PatternReco is not reproducible due to cleaning of seeds and candidate (or a bug???)
I will see if I can had a flag to make it fully reproducible at the cost of slowness and memory-blowup...

btw the current version (contrary to 7_6) will stop with

Exception Message:
Instantiating a second DetSetVector::FastFiller only one DetSetVector::FastFiller can be active at a given time!

BetterWang and others added 22 commits October 19, 2015 11:36
The thread_local in RandomNumberGeneratorService was being used to
support a backwards compatible interface. The old interface is also
supported for certain transitions via another thread_local from
the framework. It appears that the old interface is no longer being
called by non-framework supported transitions.
We no longer support calling RandomNumberGenerate::mySeed at
specific framework transitions. Test now enforces the change.
@VinInn
Copy link
Contributor Author

VinInn commented Nov 13, 2015

@cmsbuild, please test

@cmsbuild cmsbuild added this to the Next CMSSW_8_0_X milestone Nov 13, 2015
@cmsbuild
Copy link
Contributor

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/9703/console

@cmsbuild
Copy link
Contributor

#endif

#include <atomic>
#include <thread>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What symbol do you need from this include?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

none left, indeed.

@cmsbuild
Copy link
Contributor

@Dr15Jones
Copy link
Contributor

I think we should have a later pull request which gets rid of the const_cast and changes all the related code to be const so that it is explicitly const thread safe. This will allow the static analyzer to properly check the code.

@Dr15Jones
Copy link
Contributor

+1
I think at this point we need to see how this performs in a full IB.

@cmsbuild
Copy link
Contributor

static DetSetVector<T>::Item & dummy() {
assert(false);
static DetSetVector<T>::Item d; return d;
}
FastFiller(DetSetVector<T> & iv, id_type id, bool isaveEmpty=false) :
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Were all uses of FastFiller replaced with TSFastFiller? If so, then in a subsequent pull request we should remove FastFiller.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is not foreseen at least at the moment (at least on my personal side).

@cmsbuild
Copy link
Contributor

@cvuosalo
Copy link
Contributor

RecoJets /JetAnalyzers /src /myFilter.cc and RecoJets /JetAnalyzers /src /myJetAna.cc generate a lot of static analyzer warnings. Could you please fix them?

https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12411/9851/llvm-analysis/

@VinInn
Copy link
Contributor Author

VinInn commented Nov 20, 2015

@cvuosalo , I think we should survive with those (or ask jet people to take care...
v.

@VinInn
Copy link
Contributor Author

VinInn commented Nov 20, 2015

@cvuosalo , those warnings are not related to this PR...
v.

@cvuosalo
Copy link
Contributor

+1

For #12411 65c71d5

Changing DetSetNew to support concurrent fillers.. There should be no change in monitored reco quantities.

The code changes are satisfactory. There are static analyzer warnings about RecoJets/JetAnalyzers/src/myFilter.cc and RecoJets /JetAnalyzers /src /myJetAna.cc that need to be fixed in a later PR. Jenkins tests against baseline CMSSW_8_0_X_2015-11-18-2200 show no significant differences, as expected.

@cvuosalo
Copy link
Contributor

@VinInn: We are just noticing that the history of this PR includes a lot of irrelevant merges. Could you please clean it up?
Thanks.

@VinInn
Copy link
Contributor Author

VinInn commented Nov 21, 2015

@cvusalo, please give verbatim recipe on how to cleanup.
@davidlange6 If it merges w/o problems, please merge.

@davidlange6
Copy link
Contributor

@VinInn - I think @cvuosalo just means there are some commits that have nothing to do with the current request (triggering an additional signature in this case that you have been waiting for..)

davidlange6 added a commit that referenced this pull request Nov 23, 2015
Make DetSetNew to support concurrent filler
@davidlange6 davidlange6 merged commit 8957451 into cms-sw:CMSSW_8_0_X Nov 23, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants