Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plan clone refactor #1278

Merged
merged 7 commits into from Aug 13, 2020
Merged

Plan clone refactor #1278

merged 7 commits into from Aug 13, 2020

Conversation

jeffreylovitz
Copy link
Contributor

Updates ExecutionPlan Clone and Free logic to crawl the op tree rather than storing ExecutionPlan segments. Removes extraneous members from the ExecutionPlan struct.

Resolves #1270, does not resolve #1183, which should be looked into further!

Copy link
Collaborator

@swilly22 swilly22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be done quickly, let's take it online

@@ -34,7 +32,7 @@ struct ExecutionPlan {
* API for restructuring the op tree.
*/
/* Removes operation from execution plan. */
void ExecutionPlan_RemoveOp(ExecutionPlan *plan, OpBase *op);
void ExecutionPlan_RemoveOp(OpBase *op);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're so accustomed to pass the object we're working on, in this case both the plan and the op.
It feels a bit strange to have a function such as ExecutionPlan_RemoveOp which doesn't accepts an ExecutionPlan object,
although the implementation has very little to do with the plan I feel like an ASSERT(plan == op-plan) couldn't hurt and will justify restoring the original signature.

@@ -49,7 +47,7 @@ void ExecutionPlan_PushBelow(OpBase *a, OpBase *b);
void ExecutionPlan_NewRoot(OpBase *old_root, OpBase *new_root);

/* Replace a with b. */
void ExecutionPlan_ReplaceOp(ExecutionPlan *plan, OpBase *a, OpBase *b);
void ExecutionPlan_ReplaceOp(OpBase *a, OpBase *b);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as for ExecutionPlan_RemoveOp


static void _ExecutionPlan_FreeSubPlan(ExecutionPlan *plan) {
static void _ExecutionPlan_FreeSegment(ExecutionPlan *plan) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As there are no longer Segments, I think renaming this to _ExecutionPlan_FreeInternals is more suitable.

_ExecutionPlan_FreeOperations(plan->root);
plan->root = NULL;
// Free an op tree and its associated ExecutionPlan segments.
static ExecutionPlan *_ExecutionPlan_Free(OpBase *op) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename to _ExecutionPlan_FreeOpTree or something similar

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be much nicer if this function would return void and we'll detect different "segments" when traversing

Comment on lines 51 to 53
void optimizePlan(ExecutionPlan *plan) {
/* Handle UNION of execution plans. */
if(plan->is_union) {
uint segment_count = array_len(plan->segments);
for(uint i = 0; i < segment_count; i++) _optimizePlan(plan->segments[i]);
} else {
_optimizePlan(plan);
}
_optimizePlan(plan);
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for a proxy function, rename _optimizePlan to optimizePlan

*/
/* This function clones the input ExecutionPlan by recursively visiting its tree of ops.
* When an op is encountered that was constructed as part of a different ExecutionPlan segment, that segment
* and its internal members (FilterTree, record mapping, query graphs, and AST segment) are also cloned. */
ExecutionPlan *ExecutionPlan_Clone(const ExecutionPlan *template) {
if(template == NULL) return NULL;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ASSERT(template != NULL) ?

ExecutionPlan *ExecutionPlan_Clone(const ExecutionPlan *template) {
if(template == NULL) return NULL;
// Store the original AST pointer.
AST *master_ast = QueryCtx_GetAST();
// Verify that the execution plan template is not prepared yet.
assert(template->prepared == false && "Execution plan cloning should be only on templates");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use debug assert, ASSERT

// The "master" execution plan is the one constructed with the root op.
ExecutionPlan *clone = (ExecutionPlan *)clone_root->plan;
// Set the root op pointer.
clone->root = clone_root;
Copy link
Collaborator

@swilly22 swilly22 Aug 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't clone_root->plan->root is clone_root ?
in which case we can return clone ?
Redundant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This turns out to not be redundant, the root pointer is never touched by the cloning or the New op routines.

ExecutionPlan_AddOp(clone, cloned_child);
clone->record_map = raxClone(template->record_map);
if(template->ast_segment) clone->ast_segment = AST_ShallowCopy(template->ast_segment);
if(template->query_graph) clone->query_graph = QueryGraph_Clone(template->query_graph);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can avoid both query_graph and connected_components if we introduce ref count to the QueryGraph object.

swilly22
swilly22 previously approved these changes Aug 13, 2020
@swilly22 swilly22 merged commit 3d371fc into master Aug 13, 2020
@swilly22 swilly22 deleted the plan-clone-refactor branch August 13, 2020 18:14
pnxguide pushed a commit to CMU-SPEED/RedisGraph that referenced this pull request Mar 22, 2023
* WIP

* Refactor ExecutionPlan_Free

* Simplify RemoveOp signature

* Fix UNION and optimization leaks

* Address PR comments

* Revert change to Remove and Replace signatures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Changing cache logic causes test failures ExecutionPlan_Clone not producing identical plans
2 participants