Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Xrootd per site statistics #7056

Merged
merged 5 commits into from Feb 10, 2015
Merged

Xrootd per site statistics #7056

merged 5 commits into from Feb 10, 2015

Conversation

bbockelm
Copy link
Contributor

@bbockelm bbockelm commented Jan 6, 2015 

Add new storage statistics service for Xrootd.

This service keeps per-site statistics from the Xrootd layer.
Previously, the IO statistics were kept by the StatisticsSenderService
on a per-file level.  These are less-useful now that Xrootd reads
from multiple servers (and possibly multiple sites) at a time!

Currently, this has to be enabled manually and only outputs
to the FJR.  Long-term, we plan on adding additional statistics
and additional output mechanisms (correcting the now-broken
StatisticsSenderService reports we send to the Dashboard).

NOTE: there is some trickery here due to the fact that the service
registry is only available on CMSSW-managed threads.  In order to
make the service available from the callback threads, the service
pointer is cached by the request manager object.

@bbockelm
Add new storage statistics service for Xrootd.
0029af5 
This service keeps per-site statistics from the Xrootd layer.
Previously, the IO statistics were kept by the StatisticsSenderService
on a per-file level.  These are less-useful now that Xrootd reads
from multiple servers (and possibly multiple sites) at a time!

Currently, this has to be enabled manually and only outputs
to the FJR.  Long-term, we plan on adding additional statistics
and additional output mechanisms (correcting the now-broken
StatisticsSenderService reports we send to the Dashboard).

NOTE: there is some trickery here due to the fact that the service
registry is only available on CMSSW-managed threads.  In order to
make the service available from the callback threads, the service
pointer is cached by the request manager object.
@bbockelm
Integrate the statistics service with the XrdAdaptor.
de6ff2b
@cmsbuild cmsbuild added this to the Next CMSSW_7_4_X milestone Jan 6, 2015
@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 6, 2015

A new Pull Request was created by @bbockelm (Brian Bockelman) for CMSSW_7_4_X.

Xrootd per site statistics

It involves the following packages:

Utilities/XrdAdaptor

@cmsbuild, @Dr15Jones, @ktf, @nclopezo can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @wddgit this is something you requested to watch as well.
You can sign-off by replying to this message having '+1' in the first line of your reply.
You can reject by replying to this message having '-1' in the first line of your reply.
If you are a L2 or a release manager you can ask for tests by saying 'please test' in the first line of a comment.

Dr15Jones
Dr15Jones reviewed Jan 8, 2015
View reviewed changes
Utilities/XrdAdaptor/src/XrdStatistics.cc
{
edm::ParameterSetDescription desc;
desc.setComment("Report Xrootd-related statistics centrally.");
desc.addUntracked<bool>("reportToFJR", true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no such parameter used in the code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shoot, I forgot to hook that up. Will update that in the next commit or so.

@bbockelm
Address issues from code review.
76d430d 
In particular,
- We actually use the pset description to determine whether stats
  are kept.
- State information is kept in a singleton, rather than passing
  a service pointr between objects.
@cmsbuild
Copy link
Contributor

+1
Tested at: 76d430d
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7056/1804/summary.html

@cmsbuild
Copy link
Contributor

Pull request #7056 was updated. @Dr15Jones, @ktf can you please check and sign again.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7056/1804/summary.html

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 9, 2015

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs unless changes or unless it breaks tests. This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @ktf, @smuzaffar

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 9, 2015

+1
Tested at: 67357c0
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7056/2479/summary.html

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 9, 2015

This pull request is fully signed and it will be integrated in one of the next CMSSW_7_4_X IBs unless changes (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @davidlange6, @Degano, @ktf, @smuzaffar

@davidlange6
Copy link
Contributor

+1

cmsbuild added a commit that referenced this pull request Feb 10, 2015
@cmsbuild
Merge pull request #7056 from bbockelm/xrootd_per_site_statistics
a6c7122 
Xrootd per site statistics
@cmsbuild cmsbuild merged commit a6c7122 into cms-sw:CMSSW_7_4_X Feb 10, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
comparison-pending core-approved fully-signed tests-approved
Projects
None yet
Milestone
Next CMSSW_7_4_X
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@bbockelm @cmsbuild @Dr15Jones @davidlange6