[query] Fix BAD loop deoptimization with StreamAgg

[tpoterba]

CHANGELOG: Fixed bug causing poor performance and memory leaks for Matrix.annotate_rows aggregations

[danking]

bump, has a bug

[danking]

bump

[danking]

@patrick-schultz can you adopt this PR?

[patrick-schultz]

Yeah, I got it

new changes

[patrick-schultz]

@tpoterba I think I understand what was happening. I think NestingDepth wasn't handling aggregation right at all. AggLet was getting its nesting depth from the eval scope, which got compared to the nesting depth of the ref's agg scope. So even if both were at the same level inside an aggregation, we weren't forwarding. We were also ignoring StreamAggScan.

Could you check if this looks right to you?

[patrick-schultz]

I think there's another separate bug. The problem was that an ir fragment containing a StreamAgg was getting compiled, and it still had an AggLet after LowerArrayAggsToRunAggs. It's now getting inlined, but this should still work even if it isn't inlined. Maybe LowerArrayAggsToRunAggs needs to convert that to a normal Let.

[danking]

@patrick-schultz can you confirm that you anticipated this to succeed now and it just needed to be retried or something? Just a bit concerning to see a merge after two weeks without changes.

[patrick-schultz]

Yeah, I've been continually retrying. It was passing non-service tests when I approved two weeks ago.