groot/rtree: problem reading slices with formula?

[rmadar]

I am trying to read a slice, i.e a vector<float> branch with a rtree.Formula, using something like:

VarsName := []string{"hits"}, 
Fct :=: func(xs []float32) []float64 {
    res := make([]float64, len(xs))
    for i, x := range xs { res[i] = float64(x)  } 
    return res
}
f := r.FormulaFunc(VarsName, Fct)
getSlice := f.Func().(func() []float64)

In the event loop, getSlice() returns always an empty slice. Given it's quite late, it's not impossible I am doing a mistake ... But I wanted to know if there would be a reason why reading slices can fail with rtree.Formula.

[sbinet]

hum... I've added a test for that:
#738

it's working (locally, at least, let's see what the CI pipeline says).
so it might be something on your end.

how do you create the rtree.ReadVar that will be bound to the "hits" leaf?

[rmadar]

Yes ok, I am not too surprised that it works. I would need to do some more checks but to answer your question, I am using these rtree.Formula parameters:

// Branches and function
Vars := []string{"hits_time_mc"}
Fct := func(xs []float32) []float64 {
   res := make([]float64, len(xs))
   for i, x := range xs {
      res[i] = float64(x)
   }
   return res
}

// FormulFunc and value getter
formFunc := r.FormulaFunc(Vars, Fct)
getVal := formFunc.Func().(func()[]float64)

// Event loop
err = r.Read(func(ctx rtree.RCtx) error {
  fmt.Println(len(getVal())) 
  return nil
  }
)

This prints 0 for me. But my code is getting a bit convoluted ... I should start with a small standalone example - unless you see something above which is not correct.

Thanks

[sbinet]

ok. so, you pass a nil slice of rtree.ReadVar to rtree.NewReader ?
I suspect the issue comes from the way the ReadVar gets its Value.

[rmadar]

ok. so, you pass a nil slice of rtree.ReadVar to rtree.NewReader ?

Yes! I do this:

r, err := rtree.NewReader(t, []rtree.ReadVar{}, rtree.WithRange(0, ana.Nevts))

I suspect the issue comes from the way the ReadVar gets its Value.

OK ... But this still works for non-sliced types.

(sorry I closed it by mistake)

[sbinet]

must be something else.
I tried:

package main

import (
	"flag"
	"log"

	"go-hep.org/x/hep/groot"
	"go-hep.org/x/hep/groot/rtree"
)

func main() {
	flag.Parse()
	fname := flag.Arg(0)
	tname := "tree"

	f, err := groot.Open(fname)
	if err != nil {
		panic(err)
	}
	defer f.Close()

	o, err := f.Get(tname)
	if err != nil {
		panic(err)
	}
	t := o.(rtree.Tree)

	r, err := rtree.NewReader(t, []rtree.ReadVar{})
	if err != nil {
		panic(err)
	}
	defer r.Close()

	form, err := r.FormulaFunc(
		[]string{"SliF32"},
		func(xs []float32) []float64 {
			o := make([]float64, len(xs))
			for i, v := range xs {
				o[i] = float64(2 * v)
			}
			return o
		},
	)
	if err != nil {
		panic(err)
	}

	fct := form.Func().(func() []float64)

	err = r.Read(func(ctx rtree.RCtx) error {
		log.Printf("evt[%d]: %v", ctx.Entry, fct())
		return nil
	})
	if err != nil {
		panic(err)
	}
}

and then:

$> go run ./main.go /path/to/go-hep.org/x/hep/groot/testdata/leaves.root
2020/05/28 16:51:09 evt[0]: []
2020/05/28 16:51:09 evt[1]: [2]
2020/05/28 16:51:09 evt[2]: [4 4]
2020/05/28 16:51:09 evt[3]: [6 6 6]
2020/05/28 16:51:09 evt[4]: [8 8 8 8]
2020/05/28 16:51:09 evt[5]: [10 10 10 10 10]
2020/05/28 16:51:09 evt[6]: [12 12 12 12 12 12]
2020/05/28 16:51:09 evt[7]: [14 14 14 14 14 14 14]
2020/05/28 16:51:09 evt[8]: [16 16 16 16 16 16 16 16]
2020/05/28 16:51:09 evt[9]: [18 18 18 18 18 18 18 18 18]

[rmadar]

OK, we can close: it's on my side then. Thanks for checking.

[rmadar]

I am back to this. Actually there might be something odd (maybe with my input file?). I have basically copy-pasted your code, change file/tree/branch names and I get the exact same result. So I have the same problem even if I skip all my machinery, just with a simple & direct rtree.FormulaFunc.

The function to run is here: https://github.com/rmadar/tree-gonalyzer/blob/d18c7a38af67a3aaa3d9756a4c43bc83ba6f5a10/ana/example_treefunc_test.go#L189 (but this basically the same as yours).

More importantly, I have put my test file here: https://github.com/rmadar/tree-gonalyzer/blob/master/testdata/fileSlices.root

[sbinet]

ok: that's because, in your file, the "hits_mc" leaf reports no associated "LeafCount" (ie: the size of the slice/vector is 0. ROOT's TTree::Scan does display data, so there must be something missing on groot's side.)

[rmadar]

OK, then I am not crazy :)! I used this file in a couple of other ROOT codes (t->Draw(), event loop, etc ...), and I didn't see this issue. What I mean is that this file doesn't seem to be too "peculiar".

If that might help, I used the following command lines to create the branch (in a ROOT code):

vector<float> time;
tree->Branch("hits_time_mc", &time);

[sbinet]

I think I have a fix.

could you reproduce that file (but with only, say, 5 events, so its size is about ~200k ?)
I'd like to put it into my regression test suite.

[sbinet]

(and -if possible- only with the hits_n and hits_time_mc branches?)

[rmadar]

The file with only 5 events and the two branches hits_n and hits_time_mc can be downloaded here.

Just to clarfiy one point: hits_n and hits_time_mc are completely unrelated when I create the tree. I don't use one to specify the size of the second ( just save both for convenience, but I could also save only hits_time_mc and use t->Draw("@hits_time_mc.size()")). More concretely, I do:

tree->Branch("hits_n", &n, "hits_n/I"); 
tree->Branch("hits_time_mc", &hits_time_mc); 

[sbinet]

ok, thanks.

as soon as I finished my "entretien annuel", I'll commit the fix:

$> root-dump
2020/05/29 10:02:22 evt[0]: [24.412797927856445 23.422243118286133 23.469839096069336 24.914079666137695 23.116113662719727 23.13003921508789 23.375518798828125 23.057828903198242 25.786481857299805 22.85857582092285] | [12.206399 11.711122 11.73492 12.45704 11.558057 11.56502 11.687759 11.528914 12.893241 11.429288] n=10
2020/05/29 10:02:22 evt[1]: [23.436037063598633 25.970693588256836 24.462419509887695 23.650163650512695 24.811952590942383 30.67894172668457 23.878101348876953 25.87006378173828 27.323381423950195 23.939083099365234 23.786226272583008] | [11.718019 12.985347 12.23121 11.825082 12.405976 15.339471 11.939051 12.935032 13.661691 11.969542 11.893113] n=11
2020/05/29 10:02:22 evt[2]: [24.462657928466797 24.429365158081055 24.389734268188477 24.492183685302734 23.71849822998047 38.71868133544922 24.310426712036133 24.45393180847168 -9.42474365234375 23.703657150268555 23.761384963989258 23.640995025634766 23.732669830322266 26.57146644592285 -9.294095039367676] | [12.231329 12.214683 12.194867 12.246092 11.859249 19.35934 12.155213 12.226966 -4.712372 11.851829 11.8806925 11.8204975 11.866335 13.285733 -4.6470475] n=15
2020/05/29 10:02:22 evt[3]: [22.6768798828125 23.451208114624023 25.548261642456055 24.217187881469727 24.384170532226562 24.241182327270508 24.25889015197754 24.366979598999023 23.182010650634766] | [11.33844 11.725604 12.774131 12.108594 12.192085 12.120591 12.129445 12.18349 11.591005] n=9
2020/05/29 10:02:22 evt[4]: [24.312828063964844 25.28243064880371 23.35763168334961 24.659414291381836 23.15633773803711 25.025495529174805 23.680923461914062 28.24120330810547 23.750375747680664 28.266529083251953 28.211824417114258 29.810104370117188 23.62776756286621] | [12.156414 12.641215 11.678816 12.329707 11.578169 12.512748 11.840462 14.120602 11.875188 14.133265 14.105912 14.905052 11.813884] n=13

[sbinet]

thanks for the input file.
see:
#739.

[rmadar]

amazing, it works!