[10_6_X] Fixes for XRootD 5.x #41599

iarspider · 2023-05-09T14:32:32Z

PR description:

Companion PR for cms-sw/cmsdist#8484 - adopt to XRootD API changes

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #34991, #35455, #38309, #30787, #30262, and #26789

… as glibc version is used for RHEL7.

…appending triedrc=resel to opaque parameter list.

…em::Prepare() as result is expected right away and related XrdCl objects are allocated on the stack.

…bers in the handler class.

…ler.

iarspider · 2023-05-09T14:32:56Z

please test with cms-sw/cmsdist#8484

cmsbuild · 2023-05-09T14:32:58Z

A new Pull Request was created by @iarspider for CMSSW_10_6_X.

It involves the following packages:

Utilities/XrdAdaptor (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@wddgit this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

cmsbuild · 2023-05-09T15:02:54Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8cc8a0/32511/summary.html
COMMIT: 311ab5e
CMSSW: CMSSW_10_6_X_2023-05-07-0000/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41599/32511/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

XrdOucErrInfo &einfo) {return true;}
^~~~~
In file included from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/proof/proofd/inc/XrdProofdNetMgr.h:29:0,
from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/proof/proofd/src/XrdProofdAdmin.cxx:36:
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/proof/proofd/inc/XrdProofConn.h: At global scope:
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/proof/proofd/inc/XrdProofConn.h:140:4: error: 'XReqErrorType' does not name a type; did you mean 'XPErrorCode'?
XReqErrorType       LowWrite(XPClientRequest *, const void *, int);
^~~~~~~~~~~~~
XPErrorCode
[6589/8903] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin/g++   -Iinclude -D__ROOFIT_NOBANNER -Wno-implicit-fallthrough -Wno-noexcept-type -pipe -m64  -Wshadow -Wall -W -Woverloaded-virtual -fsigned-char -pthread -std=c++1z -O3 -DNDEBUG -fPIC -MD -MT graf2d/mathtext/CMakeFiles/mathtext.dir/src/mathtext.cxx.o -MF graf2d/mathtext/CMakeFiles/mathtext.dir/src/mathtext.cxx.o.d -o graf2d/mathtext/CMakeFiles/mathtext.dir/src/mathtext.cxx.o -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/graf2d/mathtext/src/mathtext.cxx
[6590/8903] /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/slc7_amd64_gcc700/external/gcc/7.0.0-pafccj/bin/g++   -Iinclude -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/core/dictgen/src -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/core/dictgen/res -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/core/dictgen/../foundation/res -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/core/dictgen/../clingutils/res -I/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/interpreter/cling/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/interpreter/llvm/src/tools/clang/include -isystem interpreter/llvm/src/tools/clang/include -isystem /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/interpreter/llvm/src/include -isystem interpreter/llvm/src/include -D__ROOFIT_NOBANNER -Wno-implicit-fallthrough -Wno-noexcept-type -pipe -m64  -Wshadow -Wall -W -Woverloaded-virtual -fsigned-char -pthread -std=c++1z -fvisibility=hidden -Wno-shadow -fno-strict-aliasing -Wno-unused-parameter -Wwrite-strings -Wno-long-long -Wno-missing-field-initializers  -DLLVM_BUILD_GLOBAL_ISEL -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -DNDEBUG -fPIC   -fno-rtti -MD -MT core/dictgen/CMakeFiles/Dictgen.dir/src/rootcling_impl.cxx.o -MF core/dictgen/CMakeFiles/Dictgen.dir/src/rootcling_impl.cxx.o.d -o core/dictgen/CMakeFiles/Dictgen.dir/src/rootcling_impl.cxx.o -c /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc700/lcg/root/6.14.09-6c945d90fec2e843d7ae58b2b477eabf/root-6.14.09/core/dictgen/src/rootcling_impl.cxx

smuzaffar · 2023-05-09T18:09:48Z

test parameters:

pull_request = Disable proofd file server for XRootD>=5 root#178, Update XRootD to 5.x in CMSSW 10_6_X cmsdist#8484

smuzaffar · 2023-05-09T18:10:03Z

please test

cmsbuild · 2023-05-09T23:28:18Z

-1

Failed Tests: Build ClangBuild
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8cc8a0/32520/summary.html
COMMIT: 311ab5e
CMSSW: CMSSW_10_6_X_2023-05-07-0000/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41599/32520/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation error when building:

>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Utilities/XrdAdaptor/src/XrdSource.cc 
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Utilities/XrdAdaptor/src/XrdRequest.cc 
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Utilities/XrdAdaptor/src/QualityMetric.cc 
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Utilities/XrdAdaptor/src/XrdStatistics.cc 
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Utilities/XrdAdaptor/src/XrdRequestManager.cc 
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Utilities/XrdAdaptor/src/XrdSource.cc:164:89: error: cannot define member function 'XrdAdaptor::Source::Source' within 'QueryAttrHandler'
 Source::Source(timespec now, std::unique_ptr fh, const std::string &exclude)
                                                                                         ^
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Utilities/XrdAdaptor/src/XrdSource.cc:165:29: error: expected unqualified-id before ')' token
     : m_lastDowngrade({0, 0}),
                             ^

Clang Build

I found compilation error while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' scram build -k -j 32 COMPILER='llvm compile'

>> Entering Package Validation/TrackerDigis
>> Entering Package Validation/TrackerHits
>> Entering Package Validation/TrackerRecHits
>> Entering Package Validation/TrackingMCTruth
>> Compile sequence completed for CMSSW CMSSW_10_6_X_2023-05-07-0000
gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 1
+ eval scram build outputlog '&&' '(python3' /data/cmsbld/jenkins/workspace/ib-run-pr-tests/cms-bot/buildLogAnalyzer.py --ignoreWarning=Wdeprecated-declarations --logDir /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/tmp/slc7_amd64_gcc700/cache/log/src '||' 'true)'
++ scram build outputlog
Reading cached build data
Entering library rule at src/Alignment/APEEstimation/plugins
>> Compiling edm plugin /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_10_6_X_2023-05-07-0000/src/Alignment/APEEstimation/plugins/ApeEstimator.cc

cmsbuild · 2023-05-10T09:37:07Z

Pull request #41599 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

iarspider · 2023-05-10T09:37:47Z

please test

cmsbuild · 2023-05-10T19:08:00Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8cc8a0/32530/summary.html
COMMIT: ef7e598
CMSSW: CMSSW_10_6_X_2023-05-07-0000/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41599/32530/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_occi_std_length had ERRORS

Comparison Summary

Summary:

You potentially removed 2 lines from the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 3215686
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3215351
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 143 log files, 102 edm output root files, 35 DQM output files
TriggerResults: no differences found

iarspider · 2023-05-10T19:30:31Z

"test_occi_std_length" is also failing in IBs

smuzaffar · 2023-05-11T08:28:14Z

@makortel @osschar these changes are needed to backport xrootd 5 in to CMSSW 10.6. Can you please review and if looks good then we can integrate

makortel · 2023-05-11T11:46:41Z

I would have preferred cherry-picking the commits rather than copying the XrdSource.cc directly. Now it also backports #38309, #30787, #30262, and #26789. They are all ok, but could you @iarspider add those in the PR description?

makortel · 2023-05-11T11:50:38Z

+core

cmsbuild · 2023-05-11T11:51:01Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (but tests are reportedly failing) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

makortel · 2023-05-11T11:51:22Z

@smuzaffar Just to be explicit, I leave the cms-sw/root#178 and cms-sw/cmsdist#8484 for you to sign.

iarspider · 2023-05-11T11:51:37Z

@makortel this was the initial attempt, but the code between 10_6 (target release for this change) and 12_1 (target release for #34991) diverged too much for cherry-picking.

smuzaffar · 2023-05-11T12:01:18Z

@makortel , sure I will take care of cmsdist/root PRs

rappoccio · 2023-05-12T13:49:43Z

+1

Externals are updating, we need this to go through IBs to clear the unsuccessful checks.

rappoccio · 2023-05-12T13:49:50Z

merge

osschar added 9 commits May 9, 2023 16:25

Use XrdSysSemaphore directly, XrdSys::LinuxSemaphore has been removed…

ce8ad72

… as glibc version is used for RHEL7.

When opening an additional file for multi-source signal intention by …

0766732

…appending triedrc=resel to opaque parameter list.

code-format

1b582f2

In XrdStorageMaker::stagein() use synchronous call to XrdCl::FileSyst…

e312e9a

…em::Prepare() as result is expected right away and related XrdCl objects are allocated on the stack.

Remove unnecessary dependencies to specific XrdCl libs.

203efc2

Revert back to async prepare call, store relevant objects as data mem…

d83e266

…bers in the handler class.

Indentation of debug msgs.

8a29e93

clang format

07fbcb2

For async calls always store XrdCl::FileSystem with the response-hand…

311ab5e

…ler.

cmsbuild added this to the CMSSW_10_6_X milestone May 9, 2023

cmsbuild added core-pending pending-signatures tests-pending orp-pending labels May 9, 2023

cmsbuild added tests-started requires-external and removed tests-pending labels May 9, 2023

cmsbuild added tests-rejected and removed tests-started labels May 9, 2023

cmsbuild added tests-started and removed tests-rejected labels May 9, 2023

smuzaffar mentioned this pull request May 9, 2023

Update XRootD to 5.x in CMSSW 10_6_X cms-sw/cmsdist#8484

Merged

cmsbuild added tests-rejected and removed tests-started labels May 9, 2023

Take XrdSource.cc from master - cherry-picking made a mess...

ef7e598

cmsbuild added tests-pending and removed tests-rejected labels May 10, 2023

cmsbuild added tests-started and removed tests-pending labels May 10, 2023

cmsbuild added tests-rejected and removed tests-started labels May 10, 2023

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels May 11, 2023

cmsbuild added orp-approved and removed orp-pending labels May 12, 2023

cmsbuild merged commit c7f1ec9 into cms-sw:CMSSW_10_6_X May 12, 2023
10 of 11 checks passed

cmsbuild mentioned this pull request May 12, 2023

Backport FastSim refiner network for NANO (DeepJet for AK4 CHS Jets) #40828

Merged

iarspider deleted the CMSSW_10_6_X branch May 15, 2023 07:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[10_6_X] Fixes for XRootD 5.x #41599

[10_6_X] Fixes for XRootD 5.x #41599

iarspider commented May 9, 2023 •

edited

iarspider commented May 9, 2023

cmsbuild commented May 9, 2023

cmsbuild commented May 9, 2023

smuzaffar commented May 9, 2023

smuzaffar commented May 9, 2023

cmsbuild commented May 9, 2023

cmsbuild commented May 10, 2023

iarspider commented May 10, 2023

cmsbuild commented May 10, 2023

iarspider commented May 10, 2023

smuzaffar commented May 11, 2023

makortel commented May 11, 2023

makortel commented May 11, 2023

cmsbuild commented May 11, 2023

makortel commented May 11, 2023

iarspider commented May 11, 2023

smuzaffar commented May 11, 2023

rappoccio commented May 12, 2023

rappoccio commented May 12, 2023

[10_6_X] Fixes for XRootD 5.x #41599

[10_6_X] Fixes for XRootD 5.x #41599

Conversation

iarspider commented May 9, 2023 • edited

PR description:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

iarspider commented May 9, 2023

cmsbuild commented May 9, 2023

cmsbuild commented May 9, 2023

External Build

smuzaffar commented May 9, 2023

smuzaffar commented May 9, 2023

cmsbuild commented May 9, 2023

Build

Clang Build

cmsbuild commented May 10, 2023

iarspider commented May 10, 2023

cmsbuild commented May 10, 2023

Unit Tests

Comparison Summary

iarspider commented May 10, 2023

smuzaffar commented May 11, 2023

makortel commented May 11, 2023

makortel commented May 11, 2023

cmsbuild commented May 11, 2023

makortel commented May 11, 2023

iarspider commented May 11, 2023

smuzaffar commented May 11, 2023

rappoccio commented May 12, 2023

rappoccio commented May 12, 2023

iarspider commented May 9, 2023 •

edited