Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance improvements in XrdFileCache #100

Merged
merged 14 commits into from
Mar 31, 2014
Merged

Conversation

alja
Copy link
Contributor

@alja alja commented Mar 28, 2014

  • Improve disk write performance
    Avoid multiple disk writes. Read from client in ram and then queue write task into global write list.
  • Fix long held locks in XrdOfsFile
    Implement virtual ioActive() method to check if destruction has to be done in a separate task.

@alja
Copy link
Contributor Author

alja commented Mar 28, 2014

Have one check to do. Temporary close.

@alja alja closed this Mar 28, 2014
@alja
Copy link
Contributor Author

alja commented Mar 28, 2014

Cksum test OK

alja@genki xrd> xrdcp -f root://xrootd.t2.ucsd.edu//store/user/alja/data60.root origin.root
[xrootd] Total 1210.40 MB |====================| 100.00 % [54.1 MB/s]
alja@genki xrd> xrdcp -f root://xrootd-proxy.t2.ucsd.edu:7777//store/user/alja/data60.root proxy-empty.root
[xrootd] Total 1210.40 MB |====================| 100.00 % [38.0 MB/s]
alja@genki xrd> xrdcp -f root://xrootd-proxy.t2.ucsd.edu:7777//store/user/alja/data60.root proxy-cached.root
[xrootd] Total 1210.40 MB |====================| 100.00 % [116.9 MB/s]
alja@genki xrd> cksum *root
519152384 1269201040 origin.root
519152384 1269201040 proxy-cached.root
519152384 1269201040 proxy-empty.root

@alja
Copy link
Contributor Author

alja commented Mar 28, 2014

600 connection test OK

Open 600 files via proxy and read them in sequence from beginning to end 2.4 MB each 10 seconds (as typical CMSSW job). This hits limit of Net traffic (1Gbs) and disk performance.

Net traffic

tr

Disk activity

The proxy started with empty cache. Performance drops as soon as file start reading from disk.
io

@alja
Copy link
Contributor Author

alja commented Mar 31, 2014

Reopen the pull request after completing two more test

  • Test TTree::Draw() on file served from file caching proxy
  • Test XrdPosixFile close -- kill 300 jobs same time
    dicsio
    discnet

@alja alja reopened this Mar 31, 2014
@ljanyst ljanyst merged commit f97e3f1 into xrootd:master Mar 31, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants