Actually fix MET cycles

[nsmith-]

Lesson learned: don't let a virtual array generator return a highlevel virtual array that uses the same cache as the parent, because then you have a strong reference to the cache in the cache = cycle. Worse, if the strong reference is inside a generator, its invisible to python gc and will never get cleaned up.

A simpler example:

import numpy as np
import awkward as ak
import weakref

form = ak.forms.Form.fromjson('"int64"')
cache = {}
def make(name):
    out = np.arange(5)
    weakref.finalize(out, print, f"all done with {name}")
    return ak.virtual(lambda: out, length=5, form=form, cache=cache)

a = ak.zip({"x": make("x"), "y": make("y")}, depth_limit=1)
b = ak.virtual(
    lambda x, y: ak.zip({"sum": x + y, "diff": x - y}, depth_limit=1),
    args=(a.x, a.y),
    length=5,
    form=ak.forms.RecordForm({"sum": form, "diff": form}),
    cache=cache,
)
del a
print("made b")

def getdiff_good(thing):
    return ak.materialized(thing.diff)

def getdiff_bad(thing):
    return thing.diff

c = ak.virtual(getdiff_good, args=(b,), length=len(b), form=form, cache=cache)
del b
print("made c")
print(f"cache size: {len(cache)}")

print(c)
del c
del cache
print("all done")

With getdiff_good, running this produces:

made b
made c
cache size: 0
[0, 0, 0, 0, 0]
all done with x
all done with y
all done

With getdiff_bad we get:

made b
made c
cache size: 0
[0, 0, 0, 0, 0]
all done
all done with y
all done with x

notice that x and y are only finalized at program exit.

[nsmith-]

One easy workaround is to use a separate cache, e.g.

c = ak.virtual(getdiff_bad, args=(b,), length=len(b), form=form, cache={})

works ok.

In practice, this means something like met_factory.build(met, corrected_jets, lazy_cache={}) as opposed to re-using the cache used to build jets.