-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
xroot: improve performances #920
Comments
I was just about to post a performance question (apparently not the same file, but the branch is exactly the same type): julia> using UnROOT
[ Info: Precompiling UnROOT [3cd96dde-e98d-4713-81e9-a4a1b0235ce9]
julia> const txrd = LazyTree("root://eospublic.cern.ch//eos/root-eos/cms_opendata_2012_nanoaod/Run2012B_DoubleMuParked.root", "Events");
julia> @time sum(txrd.nMuon[1:10^5])
1.723536 seconds (3.45 M allocations: 173.713 MiB, 2.67% gc time, 29.63% compilation time)
0x0000000000039bbb
julia> @time sum(txrd.nMuon[1:10^5])
1.174588 seconds (1.27 k allocations: 1.049 MiB)
0x0000000000039bbb
julia> const thttp = LazyTree("https://jiling.web.cern.ch/jiling/public/Run2012B_DoubleMuParked.root", "Events");
julia> @time sum(thttp.nMuon[1:10^5])
0.847792 seconds (146.22 k allocations: 9.412 MiB, 8.69% compilation time)
0x0000000000039647
julia> @time sum(thttp.nMuon[1:10^5])
0.287621 seconds (752 allocations: 1.516 MiB)
0x0000000000039647
julia> @time sum(thttp.nMuon[1:10^5])
0.279659 seconds (719 allocations: 1.516 MiB)
0x0000000000039647 both of these have the same underlying logic beyond I/O source/sink. The HTTP one is not using One thing is very different is |
This CL reduces the lock contention on xrootd.File operations. ``` $> benchstat ./ref.txt ./new.txt name old time/op new time/op delta Read-8 67.2s ± 1% 5.4s ± 7% -91.93% (p=0.000 n=9+28) name old alloc/op new alloc/op delta Read-8 343MB ± 0% 341MB ± 0% -0.78% (p=0.000 n=8+30) name old allocs/op new allocs/op delta Read-8 277k ± 0% 288k ± 0% +3.84% (p=0.000 n=7+29) ``` and now: ``` $> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null real 0m7.279s user 0m8.221s sys 0m1.256s ``` compared to: ``` $> time root-dump https://cern.ch/binet/big-file.root > /dev/null real 0m5.454s user 0m6.156s sys 0m0.228s ``` Updates go-hep#920.
This CL reduces the lock contention on xrootd.File operations. ``` $> benchstat ./ref.txt ./new.txt name old time/op new time/op delta Read-8 67.2s ± 1% 5.4s ± 7% -91.93% (p=0.000 n=9+28) name old alloc/op new alloc/op delta Read-8 343MB ± 0% 341MB ± 0% -0.78% (p=0.000 n=8+30) name old allocs/op new allocs/op delta Read-8 277k ± 0% 288k ± 0% +3.84% (p=0.000 n=7+29) ``` and now: ``` $> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null real 0m7.279s user 0m8.221s sys 0m1.256s ``` compared to: ``` $> time root-dump https://cern.ch/binet/big-file.root > /dev/null real 0m5.454s user 0m6.156s sys 0m0.228s ``` Updates go-hep#920.
#923 brought the gap of performances to:
(and that's with the so there are still some performance bits to recoup. |
this is possibly related to #399.
consider:
running over the same file but with
http[s]://
(which, currently, downloads the whole file and then serves it locally):a factor 10x is left on the floor.
(ok, not the same machine, on different networks, etc... but still)
The text was updated successfully, but these errors were encountered: