CPP: Result Set size

[kshxtij]

I'm currently running a expensive custom query to recover the callgraph of a large c codebase.

However, it errors out with Failed: Result set is larger than the limit of 2GiB.

What is this limit and is that something I can modify?

[redsun82]

Unfortunately that limit is a hard one and is not configurable. After looking it up, it seems like the unit used in the log is actually misplaced: it's not that the result set is occupying more than 2GiB of memory or storate, it's that the total number of results is exceeding MAX_ARRAY_SIZE, which is MAX_INTEGER_SIZE/2 == 2147483648, or 2 * 1024^3. Maybe working on restricting the query could help toning the result set down?

[kshxtij]

Right, that makes sense.

Im working with a path-problem query to rebuild the call graph and the result set for that is massive since I'm working with the linux kernel src. I'll look into restricting my query more or try to build the call graph more iteratively.

[redsun82]

For the linux kernel you may also have some luck with disabling options in your defconfig for parts of the code you're not interested in, to decrease the size of the database you're working with.

[intrigus-lgtm]

@redsun82 would be interesting to see whether it's possible to port the result set to panama/ffi/off-heap arrays to use bigger arrays.
If the result set is essentially just primitive values/strings that should be doable. Would make for a fun project :)

[redsun82]

@intrigus-lgtm interesting suggestion! I'll circulate it 🙂

[flyfei-cmd]

@redsun82 hi，is there any progress about solving this problem thoroughly?