Improve hoisted literals code generation

[asuhan]

Currently, we generate memory loads (from the literal buffer) when the literal node is visited and rely on loop invariant code motion pass to hoist these loads outside of the query loop. This is a heavy-handed use of LICM and might sometimes not be optimized. We should collect the constant expression with a visitor and generate the loads in the entry block.

[pressenna]

Hi, can I tackle this one?

[billmaimone]

Yes, good one for you.

[asuhan]

@psockali You can use ScalarExprVisitor to build a constant-collecting visitor. There's a pitfall though: we don't know all constants upfront when sub-queries are involved. Have a look at RelAlgTranslator::translateInOper. We can use a hybrid approach: collect and generate all the other constants upfront and leave the current handling in place (and keep LICM). Alternatively, we can reorganize the compiler to do a full pass first without emitting any IR, which we know we'll eventually have to do for other reasons as well. Going for the first approach won't hurt our ability to achieve the second and the code should be fully reusable once we have the full solution in place, so I recommend going for it as an incremental, nearly-complete solution.

[pressenna]

Hi @asuhan , I will do so.
Am I correct in assuming that only runtime constants (within the scope of a block) shall be hoisted, but literals can remain as they are?

[pressenna]

Hey @asuhan

I think I need to be much more clearer when addressing things in mapd. (e.g. literal in LLVM IR, literal in REX, literal in Analyzer::Exp, etc...).

Finally managed to look at this in more detail and it turns out that Analyzer::Constant literals are hoisted in Executor::codegenHoistedConstants into a memory buffer. And each unoptimized access to the literal requires a memory read from that buffer. As you have pointed out RelAlgTranslator::translateInOper is creating Analyzer::Constant dynamically, so scanning early on, right after translating from REX, is not very promising.

Currently I would propose to scan for Analyzer::Constant via an ScalarExprVisitor before the actual LLVM IR generation (somewhere in Executor::compileWorkUnit) and also extend the Constant class to hold a few more attributes required for this extra step.

I would then generate the memory load for each literal in @query_group_by_template and pass them as arguments to the always inlined @row_func , which also needs to be extended to take these new parameters.

I would also make this, in effect, a configurable behaviour and would probably introduce a new compiler option flag.

I hope to start tomorrow on it, if there are no objections to the above approach.

[shtilman]

Hi @psockali, your approach sounds reasonable. Add a constant scanning visitor, generate corresponding literal loads in the first block of the query func and feed these values to the always_inlined @row_func. Not sure if any Constant extensions are necessary but let's see if your motivation can be justified. No need to make this behavior configurable - we're just putting some literal loads where they should have been generated from the get go, instead of relying on LICM to move them there.

You mentioned @query_group_by_template, note that it's just one of the templates that the query func can be built from, the other one is @query_template.

[pressenna]

Hey @shtilman, as usual: theory does not reflect reality.

It turns out that during LLVM IR generation Analyzer::Constant nodes are created on the fly and sometimes the original Constant node is translated.
See Executor::codegenRowId, Executor::codegenCmpDecimalConst and Executor::codegenOuterJoinNullPlaceholder.

This invalidates the proposed approach, because scanning the source Analyzer::Exp tree will not be reliable nor sufficient.

I have to rethink .. please bear with me.

[pressenna]

Hey @shtilman,

@asuhan mentioned two approaches:

• Hybrid:
  Currently this is the approach I am going for.
• Full Pass without IR generation:
  I am not confident enough at this stage to make such a central change to the compiler as I am not sure what state(s) the compiler currently maintains and how I would reset it correctly.

I think there is an alternative way, but it would require to clone the row_function (in order to correct the function type with the hoisted variable).

Either way, let me first try the hybrid approach, it probably will be complicated enough.

[pressenna]

Hi @shtilman / @asuhan
I committed an initial version of the hybrid solution. the PR is #145 .
Please let me know if this approach is OK for now as I fear that it does not capture all the cases.

[shtilman]

Comments and discussion are in #145 PR.