[Improvement] [Java] Audit memory usage in the Java API

[narangvivek10]

To check memory usage and fix any memory leaks (if existing) in the Java API.

Assigning it to self.

[ldematte]

@narangvivek10 there are several places were we use resources.getArena(); I think these should be all examined. I started doing that for index build (especially Cagra) moving things that can be clearly short lived to use a local (confined) arena. I think there is still work to be done here, and also that some of the things should be allocated with an arena which has a more limited lifetime (e.g. the same lifetime of an index).

[chatman]

Here are some thoughts:

• We should perhaps use an arena per index lifecycle, instead of global arena (arena per cuvsResource)
• Should we use some refcounting of the arena instances to track leaks during tests?
• If yes to previous, I think we can use forbidden-apis to ban Arena.ofConfined() etc., and instead let developers only use refcounted instances of arenas.

[ldematte]

We should perhaps use an arena per index lifecycle, instead of global arena (arena per cuvsResource)

++
Or even a local arena whenever we can -- e.g. when we are allocating pointers for C functions to fill in values.
Or in the case of functions like search, we probably can have a function lifecycle.
I think the goal should be to remove the cuvsResource arena entirely (or at least do not make it public).
We may not be able to do that, but that should be our north star IMO.

Should we use some refcounting of the arena instances to track leaks during tests?
If yes to previous, I think we can use forbidden-apis to ban Arena.ofConfined() etc., and instead let developers only use refcounted instances of arenas.

I was thinking along the same lines -- we want something to trace bytes allocated and freed.
If we don't find any ready-made tool (there might be tools already doing that, e.g. using the instrumentation API profilers/debuggers use) I was thinking to do the same you say, wrap those functions and when in debug/test mode (or via some flag) enable tracing of allocations.
I think we want to do the same for function pairs like cuvsRMMAlloc/cuvsRMMFree (possibly in a AutoClosable wrapper).

[ldematte]

I went on and finished looking at the usages of the cuvsResources arena, and they can all be removed. I have a draft PR that does this, reducing the scope of Arenas either to local (lifetime of a single function) or to a single Index (tying that data to the index lifetime).

But I think it's better to review/merge #1024 (which better encapsulates the lifetime of datasets) and #1045 (which fixes a first set of lifetime issues) first.
But let me know if you'd like to proceed in a different order.