[TBB] Update to v2021.5.0 which contasins fixes for glibc-2.34 #7674

smuzaffar · 2022-03-08T08:38:24Z

oneapi-src/oneTBB#394
oneapi-src/oneTBB#519

smuzaffar · 2022-03-08T08:38:30Z

please test

cmsbuild · 2022-03-08T08:38:42Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_12_3_X/master.

@smuzaffar, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.
cms-bot commands are listed here

smuzaffar · 2022-03-08T12:39:50Z

please test

cmsbuild · 2022-03-08T19:09:22Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02b544/22935/summary.html
COMMIT: c95d108
CMSSW: CMSSW_12_3_X_2022-03-08-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7674/22935/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_edmPickEvents had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4725 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3929280
DQMHistoTests: Total failures: 636
DQMHistoTests: Total nulls: 263
DQMHistoTests: Total successes: 3928359
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -20322.154 KiB( 48 files compared)
DQMHistoSizes: changed ( 10024.0,... ): -16.692 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByVVTightIsolationMVArun2v1DBnewDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.692 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByVVTightIsolationMVArun2v1DBoldDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.692 KiB RecoTauV/hpsPFTauProducerZTT_hpsPFTauDiscriminationByVVTightIsolationMVArun2v1DBnewDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.692 KiB RecoTauV/hpsPFTauProducerZTT_hpsPFTauDiscriminationByVVTightIsolationMVArun2v1DBoldDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.612 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByMediumIsolationMVArun2v1DBnewDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.612 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByMediumIsolationMVArun2v1DBoldDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.612 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByVLooseIsolationMVArun2v1DBnewDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.612 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByVLooseIsolationMVArun2v1DBoldDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.612 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByVTightIsolationMVArun2v1DBnewDMwLT
DQMHistoSizes: changed ( 10024.0,... ): -16.612 KiB RecoTauV/hpsPFTauProducerQCD_hpsPFTauDiscriminationByVTightIsolationMVArun2v1DBoldDMwLT
DQMHistoSizes: changed ( 10024.0 ): ...
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2022-03-09T06:33:27Z

please test

cmsbuild · 2022-03-09T11:40:35Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02b544/22959/summary.html
COMMIT: c95d108
CMSSW: CMSSW_12_3_X_2022-03-08-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7674/22959/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test materialBudgetTrackerPlots had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 5638 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3703700
DQMHistoTests: Total failures: 13282
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3690395
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0 KiB( 0 files compared)
Checked 0 log files, 0 edm output root files, 49 DQM output files

smuzaffar · 2022-03-09T15:16:21Z

please test

cmsbuild · 2022-03-09T22:06:57Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02b544/22983/summary.html
COMMIT: c95d108
CMSSW: CMSSW_12_3_X_2022-03-09-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7674/22983/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695161
DQMHistoTests: Total failures: 11
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3695128
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -30288.125 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): -2541.735 KiB HLT/EGM
DQMHistoSizes: changed ( 11634.0,... ): -1786.338 KiB HLT/Tracking
DQMHistoSizes: changed ( 138.4 ): -1777.951 KiB HLT/Tracking
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2022-03-10T14:21:43Z

@makortel @Dr15Jones any objection on updating TTB to latest v2021.5.0 version?

Dr15Jones · 2022-03-10T14:38:39Z

I'm good with switching to the new version. As part of that, I'd like the task_handle API to be available. That seems to be controlled by

https://github.com/oneapi-src/oneTBB/blob/690aaf497a78a75ff72cddb084579427ab0a8ffc/include/oneapi/tbb/detail/_config.h#L523-L525

and it is unclear to me if we get it for free from __TBB_BUILD or if we need to enable TBB_PREVIEW_TASK_GROUP_EXTENSIONS.

The API is now part of the official TBB documentation so it is odd that they still treat it as a preview in the code itself.

makortel · 2022-03-10T14:39:56Z

No objection, but at this stage perhaps after 12_3_0_pre6 has been cut?

smuzaffar · 2022-03-10T21:44:42Z

please test

Dr15Jones · 2022-03-10T21:46:57Z

@smuzaffar what about TBB_PREVIEW_TASK_GROUP_EXTENSIONS?

smuzaffar · 2022-03-10T22:53:28Z

@Dr15Jones , TBB_BUILD is by default ON for this version (https://github.com/oneapi-src/oneTBB/blob/v2021.5.0/CMakeLists.txt#L101 ) so task_handle should be available with this verison

cmsbuild · 2022-03-11T05:06:32Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02b544/23028/summary.html
COMMIT: c95d108
CMSSW: CMSSW_12_3_X_2022-03-10-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7674/23028/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@cms-sw [CS9] Update py3-debugpy to version 1.5.1 #7680

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02b544/23028/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-02b544/23028/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test_edmPickEvents had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 6 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3695161
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3695125
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 48 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 204 log files, 45 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2022-03-11T21:05:16Z

+externals

cmsbuild · 2022-03-11T21:05:35Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_12_4_X/master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

Dr15Jones · 2022-04-01T16:29:30Z

@smuzaffar I tested using the new feature of TBB in CMSSW_12_4_0_pre2 (the function tbb::task_group::defer which is needed to create a tbb::task_handle ) and it failed (compiler says the function does not exist).

Therefore we do need TBB_PREVIEW_TASK_GROUP_EXTENSIONS to be set.

makortel · 2022-04-11T20:12:31Z

Therefore we do need TBB_PREVIEW_TASK_GROUP_EXTENSIONS to be set.

@smuzaffar Could we add this flag to TBB build configuraton?

smuzaffar · 2022-04-12T06:01:55Z

@Dr15Jones , can you please try USER_CXXFLAGS=-DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 scram build . If this works then we just need to update tbb.xml ( scram toolfile definition )

Dr15Jones · 2022-04-12T15:07:34Z

It works when I do.

USER_CXXFLAGS=-DTBB_PREVIEW_TASK_GROUP_EXTENSIONS scram build

[I was able to fix silly errors in my code so everything now builds using the new TBB API]

[TBB] Update to v2021.5.0 which contasins fixes for glibc-2.34

c95d108

cmsbuild added externals-pending orp-pending pending-signatures tests-started labels Mar 8, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 8, 2022

cmsbuild added tests-started and removed tests-rejected labels Mar 9, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 9, 2022

cmsbuild added tests-started and removed tests-rejected labels Mar 9, 2022

cmsbuild added tests-approved and removed tests-started labels Mar 9, 2022

cmsbuild added tests-started and removed tests-approved labels Mar 10, 2022

cmsbuild added tests-rejected and removed tests-started labels Mar 11, 2022

smuzaffar changed the base branch from IB/CMSSW_12_3_X/master to IB/CMSSW_12_4_X/master March 11, 2022 08:14

smuzaffar merged commit 401f897 into IB/CMSSW_12_4_X/master Mar 11, 2022

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Mar 11, 2022

smuzaffar deleted the smuzaffar-patch-3 branch March 11, 2022 21:32

This was referenced Mar 12, 2022

[yoda] use python3 to build #7686

Merged

Update CUDA to version 11.5.2 #7669

Merged

makortel mentioned this pull request May 16, 2022

Define TBB_PREVIEW_TASK_GROUP_EXTENSIONS #7871

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TBB] Update to v2021.5.0 which contasins fixes for glibc-2.34 #7674

[TBB] Update to v2021.5.0 which contasins fixes for glibc-2.34 #7674

smuzaffar commented Mar 8, 2022 •

edited

smuzaffar commented Mar 8, 2022

cmsbuild commented Mar 8, 2022

smuzaffar commented Mar 8, 2022

cmsbuild commented Mar 8, 2022

smuzaffar commented Mar 9, 2022

cmsbuild commented Mar 9, 2022

smuzaffar commented Mar 9, 2022

cmsbuild commented Mar 9, 2022

smuzaffar commented Mar 10, 2022 •

edited

Dr15Jones commented Mar 10, 2022

makortel commented Mar 10, 2022

smuzaffar commented Mar 10, 2022

Dr15Jones commented Mar 10, 2022

smuzaffar commented Mar 10, 2022

cmsbuild commented Mar 11, 2022

smuzaffar commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

Dr15Jones commented Apr 1, 2022

makortel commented Apr 11, 2022

smuzaffar commented Apr 12, 2022 •

edited

Dr15Jones commented Apr 12, 2022 •

edited

[TBB] Update to v2021.5.0 which contasins fixes for glibc-2.34 #7674

[TBB] Update to v2021.5.0 which contasins fixes for glibc-2.34 #7674

Conversation

smuzaffar commented Mar 8, 2022 • edited

smuzaffar commented Mar 8, 2022

cmsbuild commented Mar 8, 2022

smuzaffar commented Mar 8, 2022

cmsbuild commented Mar 8, 2022

Unit Tests

Comparison Summary

smuzaffar commented Mar 9, 2022

cmsbuild commented Mar 9, 2022

Unit Tests

Comparison Summary

smuzaffar commented Mar 9, 2022

cmsbuild commented Mar 9, 2022

Comparison Summary

smuzaffar commented Mar 10, 2022 • edited

Dr15Jones commented Mar 10, 2022

makortel commented Mar 10, 2022

smuzaffar commented Mar 10, 2022

Dr15Jones commented Mar 10, 2022

smuzaffar commented Mar 10, 2022

cmsbuild commented Mar 11, 2022

Unit Tests

Comparison Summary

smuzaffar commented Mar 11, 2022

cmsbuild commented Mar 11, 2022

Dr15Jones commented Apr 1, 2022

makortel commented Apr 11, 2022

smuzaffar commented Apr 12, 2022 • edited

Dr15Jones commented Apr 12, 2022 • edited

smuzaffar commented Mar 8, 2022 •

edited

smuzaffar commented Mar 10, 2022 •

edited

smuzaffar commented Apr 12, 2022 •

edited

Dr15Jones commented Apr 12, 2022 •

edited