Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update jemalloc to version 5.3.0 [12.4.x] #7954

Conversation

fwyzard
Copy link
Contributor

@fwyzard fwyzard commented Jun 23, 2022

This release contains many speed and space optimizations, from micro optimizations on common paths to rework of internal data structures and locking schemes, and many more too detailed to list below. Multiple percent of system level metric improvements were measured in tested production workloads.
The release has gone through large-scale production testing.

See https://github.com/jemalloc/jemalloc/releases/tag/5.3.0 for more details.

Simplify and make more uniform the syntax of the jemalloc{,-debug}.spec files.

This release contains many speed and space optimizations, from micro
optimizations on common paths to rework of internal data structures and
locking schemes, and many more too detailed to list below. Multiple percent
of system level metric improvements were measured in tested production
workloads. The release has gone through large-scale production testing.

See https://github.com/jemalloc/jemalloc/releases/tag/5.3.0 for more details.
@fwyzard
Copy link
Contributor Author

fwyzard commented Jun 23, 2022

backport #7953

@fwyzard
Copy link
Contributor Author

fwyzard commented Jun 23, 2022

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jun 23, 2022

A new Pull Request was created by @fwyzard (Andrea Bocci) for branch IB/CMSSW_12_4_X/master.

@smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.
cms-bot commands are listed here

@fwyzard
Copy link
Contributor Author

fwyzard commented Jun 23, 2022

please test for slc7_amd64_gcc10

@fwyzard
Copy link
Contributor Author

fwyzard commented Jun 23, 2022

@smuzaffar, once the tests pass, I would like to use the area created by cmsbuild to run some benchmarks and compare the performance of jemalloc 5.2.1 and 5.3.0, before merging it.

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bae14d/25720/summary.html
COMMIT: 25d2d1e
CMSSW: CMSSW_12_4_X_2022-06-22-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7954/25720/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3680359
  • DQMHistoTests: Total failures: 8
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3680329
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-bae14d/25721/summary.html
COMMIT: 25d2d1e
CMSSW: CMSSW_12_4_X_2022-06-22-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7954/25721/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 70156 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3680359
  • DQMHistoTests: Total failures: 464328
  • DQMHistoTests: Total nulls: 380
  • DQMHistoTests: Total successes: 3215629
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: -0.31499999999999995 KiB( 49 files compared)
  • DQMHistoSizes: changed ( 10224.0 ): -0.063 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 11834.0 ): -2.372 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 250202.181 ): -0.006 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 25202.0 ): 0.117 KiB SiStrip/MechanicalView
  • DQMHistoSizes: changed ( 7.3 ): 2.009 KiB SiStrip/MechanicalView
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: found differences in 14 / 49 workflows

@smuzaffar
Copy link
Contributor

+externals
@cms-sw/orp-l2 feel free to merge it for next 12.4 release.

@qliphy
Copy link
Contributor

qliphy commented Jun 28, 2022

merge

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_12_4_X/master IBs (tests are also fine). This pull request will be automatically merged.

@cmsbuild cmsbuild merged commit b522e49 into cms-sw:IB/CMSSW_12_4_X/master Jun 28, 2022
@fwyzard fwyzard deleted the IB/CMSSW_12_4_X/master_jemalloc_530 branch July 6, 2022 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants