Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid holding excessive numbers of file handles open #5080

Merged
merged 1 commit into from Aug 28, 2014

Conversation

Dr15Jones
Copy link
Contributor

When running threaded jobs with 10 threads, the job would fail because
it ran out of file handles. This change allows the code to only open
one instance of the TFile and then closes the file once all necessary
information has been read. This works since the objects pulled from
the TFile are cloned so memory ownership is transfered.

As part of the change, memory management was simplified via the use
of std::unique_ptr.

When running threaded jobs with 10 threads, the job would fail because
it ran out of file handles. This change allows the code to only open
one instance of the TFile and then closes the file once all necessary
information has been read. This works since the objects pulled from
the TFile are cloned so memory ownership is transfered.

As part of the change, memory management was simplified via the use
of std::unique_ptr.
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @Dr15Jones (Chris Jones) for CMSSW_7_2_X.

Avoid holding excessive numbers of file handles open

It involves the following packages:

RecoTauTag/RecoTau

@nclopezo, @cmsbuild, @Degano, @StoyanStoynev, @slava77 can you please review it and eventually sign? Thanks.
@jpavel this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

@StoyanStoynev
Copy link
Contributor

+1
Tested aae2b36 on top of CMSSW_7_2_X_2014-08-26-1400.
Following some indications in jenkins tests I ran extended tests - I found no differences in standard reco histograms (FWLite and DQM tests, in particular wf 202 and 38 for the latter) and no other issues. As expected. I did not test multi-threading.

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_2_X IBs unless changes (tests are also fine).

ktf added a commit that referenced this pull request Aug 28, 2014
Avoid holding excessive numbers of file handles open
@ktf ktf merged commit adcffc7 into cms-sw:CMSSW_7_2_X Aug 28, 2014
@Dr15Jones Dr15Jones deleted the avoidExcessiveNumberOfOpenFiles branch September 8, 2014 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants