Add per-site Xrootd statistics to the HTCondor monitoring. #20143
Conversation
|
A new Pull Request was created by @bbockelm (Brian Bockelman) for master. It involves the following packages: FWCore/Framework @cmsbuild, @smuzaffar, @Dr15Jones can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
-code-checks Logs: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pr-code-checks/PR-20143/73 Code check has found code style and quality issues which could be resolved by applying a patch in https://cmssdt.cern.ch/SDT/jenkins-artifacts/pr-code-checks/PR-20143/73/git-diff.patch In future, you can run |
|
please test |
|
The tests are being triggered in jenkins. |
|
+1 The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: |
|
Comparison job queued. |
|
Comparison is ready Comparison Summary:
|
Per popular demand, this will allow us to see per-site bytes read and read time, aggregated by site (as identified by the remote server; may be different from the CMS site name).
bbockelm
force-pushed
the
per_site_condor_stats
branch
from
98ecd46
to
c1f3f5a
Compare
cmsbuild
added
comparison-pending
tests-pending
and removed
comparison-available
tests-approved
labels
|
Pull request #20143 was updated. @cmsbuild, @smuzaffar, @Dr15Jones can you please check and sign again. |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pr-code-checks/PR-20143/89 |
| // Returns a mapping of "site name" to transfer statistics. The "site name" is | ||
| // as self-identified by the Xrootd host; may not necessarily match up with the | ||
| // "CMS site name". | ||
| std::map<std::string, CondorIOStats> condorUpdate(); |
There was a problem hiding this comment.
A std::vector<std::pair<std::string, CondorIOStats> would be more memory efficient and probably faster to fill.
| XrdSiteStatisticsInformation *instance = XrdSiteStatisticsInformation::getInstance(); | ||
| if (!instance) {return result;} | ||
|
|
||
| std::lock_guard<std::mutex> lock(instance->m_mutex); |
There was a problem hiding this comment.
For future: exposing a lock is a poor interface. We should rethink this service.
| @@ -23,7 +23,8 @@ namespace edm { | |||
| "AdaptorConfig", | |||
| "SiteLocalConfigService", | |||
| "StatisticsSenderService", | |||
| "CondorStatusService"}; | |||
| "CondorStatusService", | |||
| "XrdAdaptor::XrdStatisticsService"}; | |||
There was a problem hiding this comment.
I'm not overly happy about adding more default services to jobs where the framework itself does not need those services to operate.
There was a problem hiding this comment.
hi @Dr15Jones - perhaps you need two lists here - one for the things the FWK needs and one for the things that CMS wants by default?
|
please test |
|
The code-checks are being triggered in jenkins. |
|
Pull request #20143 was updated. @cmsbuild, @smuzaffar, @Dr15Jones can you please check and sign again. |
|
+code-checks |
|
@Dr15Jones - I tackled the item that had a concrete action item; however, it's not clear from your other comments what you'd like me to do (or if they're just notes for the future). |
|
please test |
|
The tests are being triggered in jenkins. |
|
+1 The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic: |
|
Comparison job queued. |
|
+1 |
cmsbuild
added
core-approved
fully-signed
and removed
core-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar (and backports should be raised in the release meeting by the corresponding L2) |
|
Comparison is ready Comparison Summary:
|
|
+1 |
Per popular demand, this will allow us to see per-site bytes read and read time, aggregated by site (as identified by the remote server; may be different from the CMS site name).
@osschar