Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve LHEReader IO (and optimize skipping events) #10287

Merged
merged 4 commits into from
Jul 28, 2015

Conversation

bbockelm
Copy link
Contributor

This branch does two things:

  1. Buffers LHEReader IO along 16MB boundaries (both compressed and uncompressed cases).
  2. Skips parsing of events that will be skipped.

When testing (2), I found there was 1.8x speedup when reading compressed files and 1.9x speedup when reading uncompressed files. This suggests, when there is efficient IO, we are dominated by XML parsing times (not decompression times).

@davidlange6 @bendavid

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @bbockelm (Brian Bockelman) for CMSSW_7_1_X.

Improve LHEReader IO (and optimize skipping events)

It involves the following packages:

GeneratorInterface/LHEInterface

@vciulli, @cmsbuild, @covarell, @bendavid, @thuer can you please review it and eventually sign? Thanks.
@mkirsano this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.
@Degano you are the release manager for this.
You can merge this pull request by typing 'merge' in the first line of your comment.

}
buffLoc_=0;
buffTotal_=rd;
std::cout << "Total buffer data read: " << buffTotal_ << std::endl;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cout should probably be removed.

@bbockelm
Copy link
Contributor Author

Good catch - development cout's removed.

@cmsbuild
Copy link
Contributor

Pull request #10287 was updated. @vciulli, @cmsbuild, @covarell, @bendavid, @thuer can you please check and sign again.

@bendavid
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_1_X IBs once checked with relvals in the development release cycle of CMSSW or unless it breaks tests. This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @smuzaffar

davidlange6 added a commit that referenced this pull request Jul 28, 2015
Improve LHEReader IO (and optimize skipping events)
@davidlange6 davidlange6 merged commit c13f150 into cms-sw:CMSSW_7_1_X Jul 28, 2015
@bbockelm
Copy link
Contributor Author

@bendavid @davidlange6 - should I do another PR for 7_6_X?

@bendavid
Copy link
Contributor

Did it already
#10431

@bbockelm
Copy link
Contributor Author

Thanks!

Is there any final summary of the performance improvements? I know the real world sometimes differs from my development VM.

I suppose just giving me a heads-up when the affected SUSY LHE gets resubmitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants