You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The transform maintains a Let-based context of Vec<DatumKnowledge> estimated for each binding that is currently in scope.
The transform then acts as a bottom up transform that both:
Computes Vec<DatumKnowledge> for the current node based on the Vec<DatumKnowledge> for its children and the current LocalId ⇒ Vec<DatumKnowledge> context, and
Uses the Vec<DatumKnowledge> for the current node to simplify its MirScalarExpr children.
LetRec implementation (basic)
Update: This was not sound as highlighted by @ggevayin this PR comment. We're abandoning this in #18323 in favor of the advanced solution sketched below.
As a first approximation, we can just:
Initialize knowledge[i][j] = DatumKnowledge::default() for each column j and CTE `i.
Using the initial estimates, descend with harvest into the original values, updating knowledge[i][j] sequentially using absorb, and then continue to the body.
LetRec implementation (advanced)
First, we need to rework DatumKnowledge to be a proper complete lattice. The proposed structure is
For the LetRec case, we try to find a fixpoint using forward analysis and the join operator of that fixpoint.
In the base case, we initialize knowledge[i][j] = DatumKnowledge::bottom() for each column j and CTE i because each CTE is initialized to the empty collection.
In the inductive case, we sequentially update knowledge[i][j] to the union of its current state and the knowledge that can be computed on a clone of values[i] using the current estimates.
We bound our attempts to at most N = min(100, 4 * C) where C is the sum of all LetRec binding arities. If no fixpoint is reached in N iterations, we set knowledge[i][j] = DatumKnowledge::top().
Once the above loop is done, we descend with harvest into the original values and then to the body using the estimated LetRec binding knowledge.
Open questions
Can the inductive case from the Implementation sketch above diverge? (cc: @mgree I think in our offline discussion earlier today you came up with some good reasons against that concern)
The text was updated successfully, but these errors were encountered:
Size estimate
L
Notes
Let
-based context ofVec<DatumKnowledge>
estimated for each binding that is currently in scope.Vec<DatumKnowledge>
for the current node based on theVec<DatumKnowledge>
for its children and the currentLocalId ⇒ Vec<DatumKnowledge>
context, andVec<DatumKnowledge>
for the current node to simplify itsMirScalarExpr
children.LetRec
implementation (basic)Update: This was not sound as highlighted by @ggevay in this PR comment. We're abandoning this in #18323 in favor of the advanced solution sketched below.
As a first approximation, we can just:
knowledge[i][j] = DatumKnowledge::default()
for each columnj
and CTE `i.harvest
into the originalvalues
, updatingknowledge[i][j]
sequentially usingabsorb
, and then continue to thebody
.LetRec
implementation (advanced)First, we need to rework
DatumKnowledge
to be a proper complete lattice. The proposed structure iswhere
and
As a diagram using the
INT
type:For the
LetRec
case, we try to find a fixpoint using forward analysis and thejoin
operator of that fixpoint.knowledge[i][j] = DatumKnowledge::bottom()
for each columnj
and CTEi
because each CTE is initialized to the empty collection.knowledge[i][j]
to the union of its current state and the knowledge that can be computed on a clone ofvalues[i]
using the current estimates.N = min(100, 4 * C)
whereC
is the sum of allLetRec
binding arities. If no fixpoint is reached inN
iterations, we setknowledge[i][j] = DatumKnowledge::top()
.Once the above loop is done, we descend with
harvest
into the originalvalues
and then to thebody
using the estimatedLetRec
binding knowledge.Open questions
The text was updated successfully, but these errors were encountered: