Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

speed up Mixins.py #26306

Merged
merged 1 commit into from
Apr 2, 2019
Merged

speed up Mixins.py #26306

merged 1 commit into from
Apr 2, 2019

Conversation

davidlange6
Copy link
Contributor

PR description:

As reported by @VinInn, start up time for MC jobs in 10_6_0_pre2 is due to a python3 compatibility change. This PR is to improve that change.

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

The code-checks are being triggered in jenkins.

@davidlange6
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-26306/9005

  • This PR adds an extra 20KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/33869/console Started: 2019/04/01 14:00

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

A new Pull Request was created by @davidlange6 (David Lange) for master.

It involves the following packages:

FWCore/ParameterSet

@cmsbuild, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @wddgit this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

Comparison job queued.

@Dr15Jones
Copy link
Contributor

@davidlange6 did you have a chance to measure the effect this change has?

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-26306/33869/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3 differences found in the comparisons
  • DQMHistoTests: Total files compared: 32
  • DQMHistoTests: Total histograms compared: 3139747
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3139549
  • DQMHistoTests: Total skipped: 197
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
  • Checked 133 log files, 14 edm output root files, 32 DQM output files

@Dr15Jones
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 1, 2019

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

@davidlange6
Copy link
Contributor Author

hi @Dr15Jones - I was hoping we ran the timing reports in the PR tests, but I guess we do not, so I don't have great numbers - it appears to save 2-3 minutes in the total time for step3 of 10224 for example.

@VinInn
Copy link
Contributor

VinInn commented Apr 1, 2019

hi @Dr15Jones - I was hoping we ran the timing reports in the PR tests, but I guess we do not, so I don't have great numbers - it appears to save 2-3 minutes in the total time for step3 of 10224 for example.

exactly what we expect!

@fabiocos
Copy link
Contributor

fabiocos commented Apr 2, 2019

@davidlange6 we do have the TimeMemorySummary activated in IBs, the problem for CPU is that it depends on the machine and on its load (although the kind of spectacular effect looked here should be anyway appreciable)

@fabiocos
Copy link
Contributor

fabiocos commented Apr 2, 2019

+1

@cmsbuild cmsbuild merged commit 45df373 into cms-sw:master Apr 2, 2019
@fabiocos
Copy link
Contributor

fabiocos commented Apr 2, 2019

not the same machine on which I did previous checks but for step3 of wf 10824.0 I now see:

TimeReport> Time report complete in 262.775 seconds
 Time Summary: 
 - Min event:   1.60077
 - Max event:   25.6009
 - Avg event:   5.36617
 - Total loop:  86.6168
 - Total init:  163.666
 - Total job:   262.775
 - EventSetup Lock:   0.000497103
 - EventSetup Get:   41.4004
 Event Throughput: 0.115451 ev/s
 CPU Summary: 
 - Total loop:  77.3087
 - Total init:  186.999
 - Total extra: 0
 - Total job:   276.078
 Processing Summary: 
 - Number of Events:  10
 - Number of Global Begin Lumi Calls:  1
 - Number of Global Begin Run Calls: 1

as expected from David's checks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants