Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xroot: improve performances #920

Open
sbinet opened this issue Feb 25, 2022 · 2 comments
Open

xroot: improve performances #920

sbinet opened this issue Feb 25, 2022 · 2 comments
Labels

Comments

@sbinet
Copy link
Member

sbinet commented Feb 25, 2022

this is possibly related to #399.

consider:

$> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null

real	1m8.557s
user	0m14.863s
sys	0m3.562s

running over the same file but with http[s]:// (which, currently, downloads the whole file and then serves it locally):

$> time root-dump https://cern.ch/binet/big-file.root > /dev/null

real	0m5.454s
user	0m6.156s
sys	0m0.228s

a factor 10x is left on the floor.
(ok, not the same machine, on different networks, etc... but still)

@sbinet sbinet added the xrootd label Feb 25, 2022
@Moelf
Copy link

Moelf commented Feb 25, 2022

I was just about to post a performance question (apparently not the same file, but the branch is exactly the same type):

julia> using UnROOT
[ Info: Precompiling UnROOT [3cd96dde-e98d-4713-81e9-a4a1b0235ce9]

julia> const txrd = LazyTree("root://eospublic.cern.ch//eos/root-eos/cms_opendata_2012_nanoaod/Run2012B_DoubleMuParked.root", "Events");

julia> @time sum(txrd.nMuon[1:10^5])
  1.723536 seconds (3.45 M allocations: 173.713 MiB, 2.67% gc time, 29.63% compilation time)
0x0000000000039bbb

julia> @time sum(txrd.nMuon[1:10^5])
  1.174588 seconds (1.27 k allocations: 1.049 MiB)
0x0000000000039bbb

julia> const thttp = LazyTree("https://jiling.web.cern.ch/jiling/public/Run2012B_DoubleMuParked.root", "Events");

julia> @time sum(thttp.nMuon[1:10^5])
  0.847792 seconds (146.22 k allocations: 9.412 MiB, 8.69% compilation time)
0x0000000000039647

julia> @time sum(thttp.nMuon[1:10^5])
  0.287621 seconds (752 allocations: 1.516 MiB)
0x0000000000039647

julia> @time sum(thttp.nMuon[1:10^5])
  0.279659 seconds (719 allocations: 1.516 MiB)
0x0000000000039647

both of these have the same underlying logic beyond I/O source/sink. The HTTP one is not using Multipart GET so it's a fair comparison. Both of these are "CERN" but evidently not the same network, before I've noticed eospublic have pretty good infrastructure for the root:// so it's telling that it's this much slow.

One thing is very different is root:// doesn't seem to benefit from async call, I wonder if it's because I'm not using Go library correctly: should ReadAt() be capable of being called in parallel?

sbinet added a commit to sbinet-hep/hep that referenced this issue Mar 8, 2022
This CL reduces the lock contention on xrootd.File operations.

```
  $> benchstat ./ref.txt ./new.txt
  name    old time/op    new time/op    delta
  Read-8     67.2s ± 1%      5.4s ± 7%  -91.93%  (p=0.000 n=9+28)

  name    old alloc/op   new alloc/op   delta
  Read-8     343MB ± 0%     341MB ± 0%   -0.78%  (p=0.000 n=8+30)

  name    old allocs/op  new allocs/op  delta
  Read-8      277k ± 0%      288k ± 0%   +3.84%  (p=0.000 n=7+29)
```

and now:
```
  $> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null

  real	0m7.279s
  user	0m8.221s
  sys	0m1.256s
```

compared to:

```
  $> time root-dump https://cern.ch/binet/big-file.root > /dev/null

  real	0m5.454s
  user	0m6.156s
  sys	0m0.228s
```

Updates go-hep#920.
sbinet added a commit to sbinet-hep/hep that referenced this issue Mar 8, 2022
This CL reduces the lock contention on xrootd.File operations.

```
  $> benchstat ./ref.txt ./new.txt
  name    old time/op    new time/op    delta
  Read-8     67.2s ± 1%      5.4s ± 7%  -91.93%  (p=0.000 n=9+28)

  name    old alloc/op   new alloc/op   delta
  Read-8     343MB ± 0%     341MB ± 0%   -0.78%  (p=0.000 n=8+30)

  name    old allocs/op  new allocs/op  delta
  Read-8      277k ± 0%      288k ± 0%   +3.84%  (p=0.000 n=7+29)
```

and now:
```
  $> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null

  real	0m7.279s
  user	0m8.221s
  sys	0m1.256s
```

compared to:

```
  $> time root-dump https://cern.ch/binet/big-file.root > /dev/null

  real	0m5.454s
  user	0m6.156s
  sys	0m0.228s
```

Updates go-hep#920.
@sbinet
Copy link
Member Author

sbinet commented Mar 8, 2022

#923 brought the gap of performances to:

  $> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null

  real	0m7.279s
  user	0m8.221s
  sys	0m1.256s

  $> time root-dump https://cern.ch/binet/big-file.root > /dev/null

  real	0m5.454s
  user	0m6.156s
  sys	0m0.228s

(and that's with the https plugin downloading the whole file locally and serving it as a local file)

so there are still some performance bits to recoup.
but the 10x drop has been fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants