decoupled result set from execution plan #929

DvirDukhan · 2020-02-19T11:06:51Z

This PR decouples result set from execution plan building. This is enabler for #907

jeffreylovitz · 2020-02-21T21:59:11Z

src/execution_plan/execution_plan.c

@@ -605,8 +603,8 @@ static void _buildMergeOp(GraphContext *gc, AST *ast, ExecutionPlan *plan,
 	AST_MergeContext merge_ctx = AST_PrepareMergeOp(clause, plan->query_graph, bound_vars);

 	// Create a Merge operation. It will store no information at this time except for any graph updates
-	// it should make due to ON MATCH and ON CREATE SET directives in the query.
-	OpBase *merge_op = NewMergeOp(plan, merge_ctx.on_match, merge_ctx.on_create, stats);
+	// it should make due to ON MATCH SET directives in the query.


This comment shouldn't be changed.

jeffreylovitz · 2020-02-21T21:59:26Z

src/execution_plan/execution_plan.c

@@ -742,10 +733,10 @@ ExecutionPlan *ExecutionPlan_UnionPlans(ResultSet *result_set, AST *ast) {
 	plan->segment_count = union_count;
 	plan->query_graph = NULL;
 	plan->record_map = raxNew();


plan->is_union = true;

i wonder why it was removed after rebase, and moreover, how the tests passed without it. I leave it for now, but it might be redundant.

I think is_union is valuable, we can use it in the optimiser phase to quickly understand if we need to apply optimisation on the different segments.

jeffreylovitz · 2020-02-21T21:59:49Z

src/execution_plan/execution_plan.c

 	AST *ast = QueryCtx_GetAST();
 	uint clause_count = cypher_ast_query_nclauses(ast->root);

 	/* Handle UNION if there are any. */
 	if(AST_ContainsClause(ast, CYPHER_AST_UNION)) {
-		return ExecutionPlan_UnionPlans(result_set, ast);
+		return ExecutionPlan_UnionPlans(ast);


jeffreylovitz · 2020-02-21T21:59:59Z

src/execution_plan/execution_plan.c

 	/* Set an exception-handling breakpoint to capture run-time errors.
 	 * encountered_error will be set to 0 when setjmp is invoked, and will be nonzero if
 	 * a downstream exception returns us to this breakpoint. */
 	int encountered_error = SET_EXCEPTION_HANDLER();

 	if(encountered_error) {
 		// Encountered a run-time error; return immediately.
-		return plan->result_set;
+		return;


jeffreylovitz · 2020-02-21T22:00:11Z

src/execution_plan/execution_plan.c

+void ExecutionPlan_PreparePlan(ExecutionPlan *plan) {
+	optimizePlan(plan);
+	QueryCtx_SetLastWriter(_ExecutionPlan_FindLastWriter(plan->root));
+	if(plan->returned_column_names) ResultSet_SetColumns(QueryCtx_GetResultSet(),


Let's have ResultSet_SetColumns invoke QueryCtx_GetResultSet rather than expecting it as an argument.

jeffreylovitz · 2020-02-21T22:00:19Z

src/execution_plan/execution_plan.c

 inline rax *ExecutionPlan_GetMappings(const ExecutionPlan *plan) {
 	assert(plan && plan->record_map);
 	return plan->record_map;
 }

+inline const char **ExecutionPlan_GetResultColumns(const ExecutionPlan *plan) {


This function seems unnecessary to me - it's only invoked at line 901.

jeffreylovitz · 2020-02-21T22:00:34Z

src/execution_plan/execution_plan.h

 	QueryGraph *query_graph;            // QueryGraph representing all graph entities in this segment.
 	FT_FilterNode *filter_tree;         // FilterTree containing filters to be applied to this segment.
 	QueryGraph **connected_components;  // Array of all connected components in this segment.
 	// NOTE - segments and segment_count are only stored for proper freeing.
 	int segment_count;                  // Number of ExecutionPlan segments.
 	ExecutionPlan **segments;           // Partial execution plans scoped to a subset of operations.
 	ObjectPool *record_pool;
+	const char **returned_column_names; // The final result set column names.


I think that column_names would be a fine name; it's not like there are other columns that aren't being returned.

jeffreylovitz · 2020-02-21T22:00:46Z

src/execution_plan/execution_plan.h

 	QueryGraph *query_graph;            // QueryGraph representing all graph entities in this segment.
 	FT_FilterNode *filter_tree;         // FilterTree containing filters to be applied to this segment.
 	QueryGraph **connected_components;  // Array of all connected components in this segment.
 	// NOTE - segments and segment_count are only stored for proper freeing.
 	int segment_count;                  // Number of ExecutionPlan segments.
 	ExecutionPlan **segments;           // Partial execution plans scoped to a subset of operations.
 	ObjectPool *record_pool;
+	const char **returned_column_names; // The final result set column names.
+	bool is_union;                      // Indicates if the execution plan as a union of execution plans.


This is currently always false, though that should be fixed by the change at 735. Do ExecutionPlans actually need to be aware of this at all, though? It looks like this is used as a switch for optimizePlan, but we can probably base the same switch on an AST check for a UNION clause - thoughts?

jeffreylovitz · 2020-02-21T22:01:50Z

src/execution_plan/execution_plan.h

@@ -88,13 +88,16 @@ OpBase *ExecutionPlan_BuildOpsFromPath(ExecutionPlan *plan, const char **vars,
 /* execution_plan.c */

 /* Creates a new execution plan from AST */
-ExecutionPlan *NewExecutionPlan(ResultSet *result_set);
+ExecutionPlan *NewExecutionPlan();


ExecutionPlan *NewExecutionPlan(void);

jeffreylovitz · 2020-02-21T22:02:55Z

src/execution_plan/execution_plan.h

+ExecutionPlan *NewExecutionPlan();
+
+/* Prepare an execution plan for execution: optimize, initialize result set schema. */
+void ExecutionPlan_PreparePlan(ExecutionPlan *plan);


I think a name like ExecutionPlan_FinalizePlan might be better in order to more clearly convey that we're between building and executing the plan.

IMO, Finalize has different semantics, such as resource and memory free.

jeffreylovitz · 2020-02-21T22:03:19Z

src/execution_plan/execution_plan.h

@@ -107,6 +110,9 @@ void ExecutionPlan_PlaceFilterOps(ExecutionPlan *plan, const OpBase *recurse_lim
 /* Retrieve the map of aliases to Record offsets in this ExecutionPlan segment. */
 rax *ExecutionPlan_GetMappings(const ExecutionPlan *plan);

+/* Retrieve the result set column names. */
+const char **ExecutionPlan_GetResultColumns(const ExecutionPlan *plan);


Even if you wish to keep this function, it doesn't need to be exposed in the header.

jeffreylovitz · 2020-02-21T22:04:02Z

src/resultset/resultset.c

-	set->columns = columns;
+	// Avoid situations like in EXPLAIN which has no result set.
+	if(!set) return;
+	// assert(columns);


Delete commented assertion.

jeffreylovitz · 2020-02-21T22:04:19Z

src/resultset/resultset.h

@@ -36,6 +36,8 @@ void ResultSet_SetColumns(ResultSet *set, const char **columns);

 int ResultSet_AddRecord(ResultSet *set, Record r);

+void ResultSet_SetColumns(ResultSet *set, const char **column_names);


Duplicate of line 35.

jeffreylovitz · 2020-02-21T22:04:41Z

src/commands/cmd_query.c

@@ -96,7 +96,7 @@ void Graph_Query(void *args) {
 	QueryCtx_SetResultSet(result_set);


IMO we may as well move the QueryCtx_SetResultSet call into NewResultSet, then have no local ResultSet variable at all in the various commands.

I prefer to avoid this "hiding" I think being explicit here is preferable,
NewResultSet should return a result-set object, let the caller decide what to do with it.

jeffreylovitz · 2020-02-21T22:06:28Z

src/commands/cmd_profile.c

@@ -61,7 +61,7 @@ void Graph_Profile(void *args) {
 	}

 	result_set = NewResultSet(ctx, false);


If not making the change suggested here, QueryCtx_SetResultSet needs to be called.

jeffreylovitz

I like this change a lot! It removes a lot of unnecessarily confusing ResultSet arguments and assignments. In fact, I think you should take it a bit further, and remove the ResultSet argument to ResultSet_SetColumns, ResultSet_Replay, ResultSet_IndexCreated, and ResultSet_IndexDeleted. Additionally, the commands (query, profile, etc) do not need to have local variables for ResultSets any more.

src/resultset/resultset.h

src/query_ctx.c

src/graph/graphcontext.c

DvirDukhan · 2020-02-23T11:56:29Z

src/execution_plan/optimizations/optimizer.c

+	// if(plan->is_union) {
+	// 	for(uint i = 0; i < plan->segment_count; i++) optimizePlan(plan->segments[i]);
+	// } else {

+	// }


optimize each segment

DvirDukhan · 2020-02-23T12:04:04Z

src/execution_plan/execution_plan.h

 	QueryGraph *query_graph;            // QueryGraph representing all graph entities in this segment.
 	FT_FilterNode *filter_tree;         // FilterTree containing filters to be applied to this segment.
 	QueryGraph **connected_components;  // Array of all connected components in this segment.
 	// NOTE - segments and segment_count are only stored for proper freeing.
 	int segment_count;                  // Number of ExecutionPlan segments.
 	ExecutionPlan **segments;           // Partial execution plans scoped to a subset of operations.
 	ObjectPool *record_pool;
+	const char **column_names;          // The final result set column names.


remove and inject column names from AST

src/execution_plan/execution_plan.h

swilly22 · 2020-02-24T07:03:52Z

src/resultset/resultset.c

@@ -206,9 +210,8 @@ void ResultSet_ReportQueryRuntime(RedisModuleCtx *ctx) {
 void ResultSet_Free(ResultSet *set) {
 	if(!set) return;

-	array_free(set->columns);
+	if(set->columns) array_free(set->columns);
 	rm_free(set->columns_record_map);


Similar to the change at line 213, add if(set->columns_record_map)

swilly22 · 2020-02-24T07:04:51Z

src/resultset/resultset.c

-	assert(set && columns);
-	set->columns = columns;
+void ResultSet_SetColumns(const char **columns) {
+	ResultSet *set = QueryCtx_GetResultSet();


I don't like this design where we call ResultSet_* functions without passing a result-set object,
please revert.

swilly22 · 2020-02-24T07:07:45Z

src/execution_plan/optimizations/optimizer.c

+	/* Handle UNION if there are any. */
+	if(AST_ContainsClause(ast, CYPHER_AST_UNION)) {
+		for(uint i = 0; i < plan->segment_count; i++) _optimizePlan(plan->segments[i]);
+	} else {
+		_optimizePlan(plan);
+	}


Does plan->segments[0] points to plan by any chance?
if so all we need is: for(uint i = 0; i < plan->segment_count; i++) _optimizePlan(plan->segments[i]);

Also are there other situations where plan->segment_count > 0 except UNION?
How would you define an execution plan segment?

in UNION situations, segments do not hold the plan itself
in regular situations, the plan is in index segments_count +1

from NewExecutionPlan

for(int i = 0; i < segment_count; i++) { uint end_offset = segment_indices[i]; // Slice the AST to only include the clauses in the current segment. AST *ast_segment = AST_NewSegment(ast, start_offset, end_offset); ast_segments[i] = ast_segment; // Construct a new ExecutionPlanSegment. ExecutionPlan *segment = ExecutionPlan_NewEmptyExecutionPlan(); ExecutionPlan_PopulateExecutionPlan(segment); segments[i] = segment; start_offset = end_offset; }

and

// Disregard self. plan->segment_count = segment_count - 1; plan->segments = segments;

swilly22 · 2020-02-24T07:09:10Z

src/execution_plan/optimizations/optimizer.c

-
-	// Tries to compact filter trees, and remove redundant filters.
+void _optimizePlan(ExecutionPlan *plan) {
+// Tries to compact filter trees, and remove redundant filters.


swilly22 · 2020-02-24T07:12:27Z

src/execution_plan/execution_plan.h

 	QueryGraph *query_graph;            // QueryGraph representing all graph entities in this segment.
 	FT_FilterNode *filter_tree;         // FilterTree containing filters to be applied to this segment.
 	QueryGraph **connected_components;  // Array of all connected components in this segment.
 	// NOTE - segments and segment_count are only stored for proper freeing.
 	int segment_count;                  // Number of ExecutionPlan segments.
 	ExecutionPlan **segments;           // Partial execution plans scoped to a subset of operations.
 	ObjectPool *record_pool;
+	bool is_exec_ready;                 // Indicates if the execution plan is ready for execute.


Rename to bool prepared

swilly22 · 2020-02-24T07:23:01Z

src/execution_plan/execution_plan.c

-		assert(plan->root->type == OPType_PROC_CALL);
-		OpProcCall *last_op = (OpProcCall *)plan->root;
+		OpBase *proc_call_base = ExecutionPlan_LocateFirstOp(plan->root, OPType_PROC_CALL);
+		assert(proc_call_base && (proc_call_base->type == OPType_PROC_CALL));


proc_call_base->type == OPType_PROC_CALL redundant, you've asked for: ExecutionPlan_LocateFirstOp(plan->root, OPType_PROC_CALL);

swilly22 · 2020-02-24T07:25:09Z

src/execution_plan/execution_plan.c

+	return plan;
+}
+
+static void _ExecutionPlan_SetResultColumns(ExecutionPlan *plan) {


It seems like this logic is a bit miss placed,
To me it makes more sense to place setting of the result-set columns within ResultSet_New.

swilly22 · 2020-02-24T07:25:37Z

src/execution_plan/execution_plan.c


-	// Optimize the operations in the ExecutionPlan.
+void ExecutionPlan_PreparePlan(ExecutionPlan *plan) {
+	_ExecutionPlan_SetResultColumns(plan);


I hope we'll be able to avoid this call, see my comment above.

swilly22 · 2020-02-24T07:27:44Z

src/execution_plan/execution_plan.c

+clean_up:
+	return QueryCtx_GetResultSet();


There's no real clean_up logic here, maybe we should revert this change.

swilly22 · 2020-02-24T07:30:51Z

src/execution_plan/execution_plan.c

-ResultSet *ExecutionPlan_Profile(ExecutionPlan *plan) {
+void ExecutionPlan_Profile(ExecutionPlan *plan) {
 	_ExecutionPlan_InitProfiling(plan->root);
-	ResultSet *rs = ExecutionPlan_Execute(plan);
+	ExecutionPlan_Execute(plan);
 	_ExecutionPlan_FinalizeProfiling(plan->root);
-	return rs;
 }


Similar to ExecutionPlan_Execute ExecutionPlan_Profile should return a result-set.
the caller is expected to free it.

* Support float inputs for modulo computations (#895) * Add references * Update mkdocs.yml * Update References.md * fixed ast mapping for path filter (#896) * fixed ast mapping for path filter * renamed tests. open a new redis graph client per tests Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added mutex per matrix (#898) * added mutex per matrix * fixed PR comments * updated license headers to 2020 (#902) * Preserve order of op's children array when introducing index scans (#912) * Fix memory leaks (#917) * Fix memory leaks * Use original logic for MarkWriter * added filter tree clone (#915) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Optimize cp (#906) * added optimize cp * fixed a bug. added test * add optimization_util files * fixed PR comments * fixed PR comments * added multiple branch cp optimization * fixed PR comment * added in place replacement at ExecutionPlan_RemoveOp * added test for cp optimization and semi apply * fixed PR comments * Added non existing entity runtime error (#919) * Added non existing entity runtime error * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * RediSearch query error reporting (#925) * added filter tree compaction (#922) * added filter tree compaction * fixed PR comments * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Handle both possible execution orders in concurrent rename test (#926) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Refferencing a RedisGraph javascript client library (#928) * Improve client libraries sections on both readme and clients.md Clients libraries are subject to repetitive edition, it seems better to have a concise formatting * refferencing my own contribution to clients libraries Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Rust client (#931) * Update README.md * Update clients.md * intoduced fpClone in OpBase (#930) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Apply deplete match stream (#913) * Deplete Apply op's match stream for every left-hand Record * Disallow ExpandInto ops on variables beneath Apply ops * Add tests * PR fixes * Argument op holds one Record * PR fixes * Improve logic for building OpArgument modifies arrays * standardize logic for building tmp ExecutionPlans * Remove sub-ExecutionPlan logic * Fix CondTraverseReset routine * Fix ExpandIntoReset * Fix variable-length QueryGraph pointer * Always reset match branch * Add explanatory comment * PR fixes * PR fixes * raxValues returns void pointer array Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * increased allowed parameters count to UNIT_MAX. fixed a bug in query_ctx (#933) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * introduce mapping between record entries and projected columns (#936) * added arr clone with cb (#937) * added arr clone with cb * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Leak fixes (#935) * Close Redis key handles * Fix memory leak in checking whether procedures are read-only * Improve explanatory comment Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * label matrix should be fetch right before eval (#938) * More sensible function signature for SIValue_Free (#941) * Resolve memory leaks on Path SIValues (#940) * Resolve memory leaks on Path SIValues * Update op_delete.c * Update op_cond_var_len_traverse.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GraphBLAS 3.2.0 (#942) * GraphBLAS 3.2.0 * updated makefiles * Remove unnecessary GraphBLAS build flag Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Simplify OpAggregate logic, remove unnecessary struct members (#947) * Simplify OpAggregate logic, remove unnecessary struct members * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add compile-time error for unsupported AST node types (#944) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * removed unused operations and types (#950) * remove GraphBLAS cacheing from CI (#953) * switch from OR AND semiring to ANY PAIR (#955) * using the structured semiring we can combine relation and relation ma… (#964) * using the structured semiring we can combine relation and relation mapping into a single matrix * address PR comments * Update graph.c * decoupled result set from execution plan (#929) * decoupled result set from execution plan * after rebase merge * fixed PR comments * Restored resulte set * fixed PR comments * fixed PR comments * fixed PR comments * Update ast.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Always free emptied space on AlgebraicExpression replacement (#986) * Fix leak on full-text index queries with syntax error (#985) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Fix leak on index scan IN optimization (#984) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Minor updates to bulk deletion (#994) * Remove redundant clone from bulk insertion of string props (#993) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Fix memory leaks on RDB-loaded strings (#991) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add flow test to validate (#967) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Update outdated logic for aggregate groups (#968) * Update outdated logic for aggregate groups * Update group.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added all_node_scan clone (#960) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added op argument clone (#959) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added cartesian product clone (#963) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added expand into clone (#973) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added project op clone (#980) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added skip op clone (#983) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added op unwind clone (#988) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added limit op clone (#976) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added proc call clone (#979) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added filter op clone (#974) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added delete op clone (#970) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added distinct op clone (#972) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * add hash join op clone (#989) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added semi apply clone (#982) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added results op clone (#981) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added join op clone (#975) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added apply multiplexer clone (#961) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added cond var len traverse clone (#966) * added cond var len traverse clone * added conditional traverse clone * fixed PR comments * Reintroduce logic for freeing memory on run-time errors (#992) * Reintroduce logic for freeing memory on run-time errors * Remove VolatileRecord logic for freeing after run-time errors * Fix memory leaks on run-time errors in OpProject Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Free internal edge arrays on relationship matrix deletion (#997) * Free internal edge arrays on relationship matrix deletion * Update graph.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add debug function to print query (#995) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added clone for create, merge, merge_create, update (#969) * added clone for create, merge, merge_create, update * Update ast_shared.c * Update ast_shared.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Agg func fixes (#946) * Fix memory leaks in Collect function * Fix leak in aggregate func's SIValue result * Fix leak in children of aggregate function call * Simplify variable-length path free logic * Improve ownership logic in Collect, add explanatory comments * PR fix Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added label scan clone (#978) * added label scan clone * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added id seek clone (#977) * added id seek clone * added better comment on the range clone * Update op_node_by_id_seek.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added op aggreage clone (#958) * added op aggreage clone * fixed PR comments * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add install instructions for OpenMP (#1006) * Add install instructions for OpenMP * Update README * added op sort clone (#987) * added op sort clone * fixed PR comments * removed free_list logic * Update execution_plan.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * re-enable graph.profile (#1004) * Re-enable GraphBLAS circle-ci cache (#1007) * Fix leak on projected heap-allocated graph entities (#996) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GRAPH.SLOWLOG (#897) * slowlog WIP * slowlog per graph * updated docs * Add longer-running query to slowlog flow test * avoid race, log only GRAPH.QUERY Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Use NOP label scan if range iterator construction fails (#1001) * Use NOP label scan if range iterator construction fails * deplete iterator for invalid range * dont access op consume function directly Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * removed redisearch optimization (#1010) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Note ReplicaOf memory constraint in Rampfile (#1011) * Automate leak check (#1002) * Add make memcheck rule * Add Circle memcheck job * Don't use Docker image for automated leak checking * Enable log names by migrating flow test Env initialization * Disable invalid TCK scenarios of issue #945 * Run memory test without optimizations after packaging artifacts * Add suppressions for leaks in flush-vs-shutdown race Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Write longer-running query for slowlog test (#1012) * OPType bitmask switched to contiguous enum (#1013) * OPType bitmask switched to contiguous enum * PR fixes * Clean up op tree modification code * Add static array of all scan ops * Remove unnecessary conditional * Remove static type array for unoptimized scans * Validate arity per-command rather than globally (#1023) * Ar exp param (#990) * wip * wip * after rebase * updated libcypher parser * added AR_EXP_PARAM * fixed PR comments * fixed PR comments * fixed PR comments * fixed PR comments * Update arithmetic_expression.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Fix invalid op sequence when introducing index scans (#1028) * Fix invalid op sequence when partially replacing filters with index scans * Simplify op-freeing logic in utilizeIndices * Automate testing against enterprise v5.4.14 * push down transpose operations (#1032) * push down transpose operations * added test for transposed bi-directional edge * Update algebraic_expression_optimization.c * allow skip and limit to be parametes. (#1020) * wip * test pass * added skip limit params test * added failure test * fixed PR comments * fixed online review comments * restored traverse record cap * fixing memory leaks * fixed PR comments * changed redisearch version to 1.6.11 (#1035) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * version bump * version bump * Version 2.0.2 (#949) * Support float inputs for modulo computations (#895) * Add references * Update mkdocs.yml * Update References.md * fixed ast mapping for path filter (#896) * fixed ast mapping for path filter * renamed tests. open a new redis graph client per tests Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added mutex per matrix (#898) * added mutex per matrix * fixed PR comments * updated license headers to 2020 (#902) * Preserve order of op's children array when introducing index scans (#912) * Fix memory leaks (#917) * Fix memory leaks * Use original logic for MarkWriter * added filter tree clone (#915) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Optimize cp (#906) * added optimize cp * fixed a bug. added test * add optimization_util files * fixed PR comments * fixed PR comments * added multiple branch cp optimization * fixed PR comment * added in place replacement at ExecutionPlan_RemoveOp * added test for cp optimization and semi apply * fixed PR comments * Added non existing entity runtime error (#919) * Added non existing entity runtime error * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * RediSearch query error reporting (#925) * added filter tree compaction (#922) * added filter tree compaction * fixed PR comments * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Handle both possible execution orders in concurrent rename test (#926) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Refferencing a RedisGraph javascript client library (#928) * Improve client libraries sections on both readme and clients.md Clients libraries are subject to repetitive edition, it seems better to have a concise formatting * refferencing my own contribution to clients libraries Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Rust client (#931) * Update README.md * Update clients.md * intoduced fpClone in OpBase (#930) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Apply deplete match stream (#913) * Deplete Apply op's match stream for every left-hand Record * Disallow ExpandInto ops on variables beneath Apply ops * Add tests * PR fixes * Argument op holds one Record * PR fixes * Improve logic for building OpArgument modifies arrays * standardize logic for building tmp ExecutionPlans * Remove sub-ExecutionPlan logic * Fix CondTraverseReset routine * Fix ExpandIntoReset * Fix variable-length QueryGraph pointer * Always reset match branch * Add explanatory comment * PR fixes * PR fixes * raxValues returns void pointer array Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * increased allowed parameters count to UNIT_MAX. fixed a bug in query_ctx (#933) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * introduce mapping between record entries and projected columns (#936) * added arr clone with cb (#937) * added arr clone with cb * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Leak fixes (#935) * Close Redis key handles * Fix memory leak in checking whether procedures are read-only * Improve explanatory comment Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * label matrix should be fetch right before eval (#938) * More sensible function signature for SIValue_Free (#941) * Resolve memory leaks on Path SIValues (#940) * Resolve memory leaks on Path SIValues * Update op_delete.c * Update op_cond_var_len_traverse.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GraphBLAS 3.2.0 (#942) * GraphBLAS 3.2.0 * updated makefiles * Remove unnecessary GraphBLAS build flag Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Simplify OpAggregate logic, remove unnecessary struct members (#947) * Simplify OpAggregate logic, remove unnecessary struct members * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add compile-time error for unsupported AST node types (#944) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * version bump 2.0.2 Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> Co-authored-by: Guy Korland <gkorland@gmail.com> Co-authored-by: DvirDukhan <dvir@redislabs.com> Co-authored-by: Sceat <11330271+Sceat@users.noreply.github.com> * Version 2.0.3 (#951) * Support float inputs for modulo computations (#895) * Add references * Update mkdocs.yml * Update References.md * fixed ast mapping for path filter (#896) * fixed ast mapping for path filter * renamed tests. open a new redis graph client per tests Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added mutex per matrix (#898) * added mutex per matrix * fixed PR comments * updated license headers to 2020 (#902) * Preserve order of op's children array when introducing index scans (#912) * Fix memory leaks (#917) * Fix memory leaks * Use original logic for MarkWriter * added filter tree clone (#915) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Optimize cp (#906) * added optimize cp * fixed a bug. added test * add optimization_util files * fixed PR comments * fixed PR comments * added multiple branch cp optimization * fixed PR comment * added in place replacement at ExecutionPlan_RemoveOp * added test for cp optimization and semi apply * fixed PR comments * Added non existing entity runtime error (#919) * Added non existing entity runtime error * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * RediSearch query error reporting (#925) * added filter tree compaction (#922) * added filter tree compaction * fixed PR comments * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Handle both possible execution orders in concurrent rename test (#926) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Refferencing a RedisGraph javascript client library (#928) * Improve client libraries sections on both readme and clients.md Clients libraries are subject to repetitive edition, it seems better to have a concise formatting * refferencing my own contribution to clients libraries Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Rust client (#931) * Update README.md * Update clients.md * intoduced fpClone in OpBase (#930) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Apply deplete match stream (#913) * Deplete Apply op's match stream for every left-hand Record * Disallow ExpandInto ops on variables beneath Apply ops * Add tests * PR fixes * Argument op holds one Record * PR fixes * Improve logic for building OpArgument modifies arrays * standardize logic for building tmp ExecutionPlans * Remove sub-ExecutionPlan logic * Fix CondTraverseReset routine * Fix ExpandIntoReset * Fix variable-length QueryGraph pointer * Always reset match branch * Add explanatory comment * PR fixes * PR fixes * raxValues returns void pointer array Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * increased allowed parameters count to UNIT_MAX. fixed a bug in query_ctx (#933) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * introduce mapping between record entries and projected columns (#936) * added arr clone with cb (#937) * added arr clone with cb * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Leak fixes (#935) * Close Redis key handles * Fix memory leak in checking whether procedures are read-only * Improve explanatory comment Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * label matrix should be fetch right before eval (#938) * More sensible function signature for SIValue_Free (#941) * Resolve memory leaks on Path SIValues (#940) * Resolve memory leaks on Path SIValues * Update op_delete.c * Update op_cond_var_len_traverse.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GraphBLAS 3.2.0 (#942) * GraphBLAS 3.2.0 * updated makefiles * Remove unnecessary GraphBLAS build flag Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Simplify OpAggregate logic, remove unnecessary struct members (#947) * Simplify OpAggregate logic, remove unnecessary struct members * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add compile-time error for unsupported AST node types (#944) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * removed unused operations and types (#950) * version bump 2.0.3 Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> Co-authored-by: Guy Korland <gkorland@gmail.com> Co-authored-by: DvirDukhan <dvir@redislabs.com> Co-authored-by: Sceat <11330271+Sceat@users.noreply.github.com> * V2.0.4 (#954) * Support float inputs for modulo computations (#895) * Add references * Update mkdocs.yml * Update References.md * fixed ast mapping for path filter (#896) * fixed ast mapping for path filter * renamed tests. open a new redis graph client per tests Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added mutex per matrix (#898) * added mutex per matrix * fixed PR comments * updated license headers to 2020 (#902) * Preserve order of op's children array when introducing index scans (#912) * Fix memory leaks (#917) * Fix memory leaks * Use original logic for MarkWriter * added filter tree clone (#915) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Optimize cp (#906) * added optimize cp * fixed a bug. added test * add optimization_util files * fixed PR comments * fixed PR comments * added multiple branch cp optimization * fixed PR comment * added in place replacement at ExecutionPlan_RemoveOp * added test for cp optimization and semi apply * fixed PR comments * Added non existing entity runtime error (#919) * Added non existing entity runtime error * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * RediSearch query error reporting (#925) * added filter tree compaction (#922) * added filter tree compaction * fixed PR comments * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Handle both possible execution orders in concurrent rename test (#926) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Refferencing a RedisGraph javascript client library (#928) * Improve client libraries sections on both readme and clients.md Clients libraries are subject to repetitive edition, it seems better to have a concise formatting * refferencing my own contribution to clients libraries Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Rust client (#931) * Update README.md * Update clients.md * intoduced fpClone in OpBase (#930) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Apply deplete match stream (#913) * Deplete Apply op's match stream for every left-hand Record * Disallow ExpandInto ops on variables beneath Apply ops * Add tests * PR fixes * Argument op holds one Record * PR fixes * Improve logic for building OpArgument modifies arrays * standardize logic for building tmp ExecutionPlans * Remove sub-ExecutionPlan logic * Fix CondTraverseReset routine * Fix ExpandIntoReset * Fix variable-length QueryGraph pointer * Always reset match branch * Add explanatory comment * PR fixes * PR fixes * raxValues returns void pointer array Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * increased allowed parameters count to UNIT_MAX. fixed a bug in query_ctx (#933) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * introduce mapping between record entries and projected columns (#936) * added arr clone with cb (#937) * added arr clone with cb * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Leak fixes (#935) * Close Redis key handles * Fix memory leak in checking whether procedures are read-only * Improve explanatory comment Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * label matrix should be fetch right before eval (#938) * More sensible function signature for SIValue_Free (#941) * Resolve memory leaks on Path SIValues (#940) * Resolve memory leaks on Path SIValues * Update op_delete.c * Update op_cond_var_len_traverse.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GraphBLAS 3.2.0 (#942) * GraphBLAS 3.2.0 * updated makefiles * Remove unnecessary GraphBLAS build flag Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Simplify OpAggregate logic, remove unnecessary struct members (#947) * Simplify OpAggregate logic, remove unnecessary struct members * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add compile-time error for unsupported AST node types (#944) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * removed unused operations and types (#950) * remove GraphBLAS cacheing from CI (#953) * version bump Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> Co-authored-by: Guy Korland <gkorland@gmail.com> Co-authored-by: DvirDukhan <dvir@redislabs.com> Co-authored-by: Sceat <11330271+Sceat@users.noreply.github.com> * V2.0.5 (#957) * Support float inputs for modulo computations (#895) * Add references * Update mkdocs.yml * Update References.md * fixed ast mapping for path filter (#896) * fixed ast mapping for path filter * renamed tests. open a new redis graph client per tests Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added mutex per matrix (#898) * added mutex per matrix * fixed PR comments * updated license headers to 2020 (#902) * Preserve order of op's children array when introducing index scans (#912) * Fix memory leaks (#917) * Fix memory leaks * Use original logic for MarkWriter * added filter tree clone (#915) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Optimize cp (#906) * added optimize cp * fixed a bug. added test * add optimization_util files * fixed PR comments * fixed PR comments * added multiple branch cp optimization * fixed PR comment * added in place replacement at ExecutionPlan_RemoveOp * added test for cp optimization and semi apply * fixed PR comments * Added non existing entity runtime error (#919) * Added non existing entity runtime error * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * RediSearch query error reporting (#925) * added filter tree compaction (#922) * added filter tree compaction * fixed PR comments * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Handle both possible execution orders in concurrent rename test (#926) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Refferencing a RedisGraph javascript client library (#928) * Improve client libraries sections on both readme and clients.md Clients libraries are subject to repetitive edition, it seems better to have a concise formatting * refferencing my own contribution to clients libraries Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Rust client (#931) * Update README.md * Update clients.md * intoduced fpClone in OpBase (#930) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Apply deplete match stream (#913) * Deplete Apply op's match stream for every left-hand Record * Disallow ExpandInto ops on variables beneath Apply ops * Add tests * PR fixes * Argument op holds one Record * PR fixes * Improve logic for building OpArgument modifies arrays * standardize logic for building tmp ExecutionPlans * Remove sub-ExecutionPlan logic * Fix CondTraverseReset routine * Fix ExpandIntoReset * Fix variable-length QueryGraph pointer * Always reset match branch * Add explanatory comment * PR fixes * PR fixes * raxValues returns void pointer array Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * increased allowed parameters count to UNIT_MAX. fixed a bug in query_ctx (#933) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * introduce mapping between record entries and projected columns (#936) * added arr clone with cb (#937) * added arr clone with cb * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Leak fixes (#935) * Close Redis key handles * Fix memory leak in checking whether procedures are read-only * Improve explanatory comment Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * label matrix should be fetch right before eval (#938) * More sensible function signature for SIValue_Free (#941) * Resolve memory leaks on Path SIValues (#940) * Resolve memory leaks on Path SIValues * Update op_delete.c * Update op_cond_var_len_traverse.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GraphBLAS 3.2.0 (#942) * GraphBLAS 3.2.0 * updated makefiles * Remove unnecessary GraphBLAS build flag Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Simplify OpAggregate logic, remove unnecessary struct members (#947) * Simplify OpAggregate logic, remove unnecessary struct members * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add compile-time error for unsupported AST node types (#944) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * removed unused operations and types (#950) * remove GraphBLAS cacheing from CI (#953) * switch from OR AND semiring to ANY PAIR (#955) * version bump Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> Co-authored-by: Guy Korland <gkorland@gmail.com> Co-authored-by: DvirDukhan <dvir@redislabs.com> Co-authored-by: Sceat <11330271+Sceat@users.noreply.github.com> * rebase master to v2.0.5. Changed version to 2.0.6 * V2.0.7 * V2.0.8 * V2.0.9 * version bump Co-authored-by: Guy Korland <gkorland@gmail.com> Co-authored-by: DvirDukhan <dvir@redislabs.com> Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> Co-authored-by: Sceat <11330271+Sceat@users.noreply.github.com> Co-authored-by: Pieter Cailliau <pieter@redislabs.com>

* Support float inputs for modulo computations (#895) * Add references * Update mkdocs.yml * Update References.md * fixed ast mapping for path filter (#896) * fixed ast mapping for path filter * renamed tests. open a new redis graph client per tests Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added mutex per matrix (#898) * added mutex per matrix * fixed PR comments * updated license headers to 2020 (#902) * Preserve order of op's children array when introducing index scans (#912) * Fix memory leaks (#917) * Fix memory leaks * Use original logic for MarkWriter * added filter tree clone (#915) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Optimize cp (#906) * added optimize cp * fixed a bug. added test * add optimization_util files * fixed PR comments * fixed PR comments * added multiple branch cp optimization * fixed PR comment * added in place replacement at ExecutionPlan_RemoveOp * added test for cp optimization and semi apply * fixed PR comments * Added non existing entity runtime error (#919) * Added non existing entity runtime error * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * RediSearch query error reporting (#925) * added filter tree compaction (#922) * added filter tree compaction * fixed PR comments * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Handle both possible execution orders in concurrent rename test (#926) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Refferencing a RedisGraph javascript client library (#928) * Improve client libraries sections on both readme and clients.md Clients libraries are subject to repetitive edition, it seems better to have a concise formatting * refferencing my own contribution to clients libraries Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Rust client (#931) * Update README.md * Update clients.md * intoduced fpClone in OpBase (#930) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Apply deplete match stream (#913) * Deplete Apply op's match stream for every left-hand Record * Disallow ExpandInto ops on variables beneath Apply ops * Add tests * PR fixes * Argument op holds one Record * PR fixes * Improve logic for building OpArgument modifies arrays * standardize logic for building tmp ExecutionPlans * Remove sub-ExecutionPlan logic * Fix CondTraverseReset routine * Fix ExpandIntoReset * Fix variable-length QueryGraph pointer * Always reset match branch * Add explanatory comment * PR fixes * PR fixes * raxValues returns void pointer array Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * increased allowed parameters count to UNIT_MAX. fixed a bug in query_ctx (#933) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * introduce mapping between record entries and projected columns (#936) * added arr clone with cb (#937) * added arr clone with cb * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Leak fixes (#935) * Close Redis key handles * Fix memory leak in checking whether procedures are read-only * Improve explanatory comment Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * label matrix should be fetch right before eval (#938) * More sensible function signature for SIValue_Free (#941) * Resolve memory leaks on Path SIValues (#940) * Resolve memory leaks on Path SIValues * Update op_delete.c * Update op_cond_var_len_traverse.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GraphBLAS 3.2.0 (#942) * GraphBLAS 3.2.0 * updated makefiles * Remove unnecessary GraphBLAS build flag Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Simplify OpAggregate logic, remove unnecessary struct members (#947) * Simplify OpAggregate logic, remove unnecessary struct members * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add compile-time error for unsupported AST node types (#944) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * removed unused operations and types (#950) * remove GraphBLAS cacheing from CI (#953) * switch from OR AND semiring to ANY PAIR (#955) * using the structured semiring we can combine relation and relation ma… (#964) * using the structured semiring we can combine relation and relation mapping into a single matrix * address PR comments * Update graph.c * decoupled result set from execution plan (#929) * decoupled result set from execution plan * after rebase merge * fixed PR comments * Restored resulte set * fixed PR comments * fixed PR comments * fixed PR comments * Update ast.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Always free emptied space on AlgebraicExpression replacement (#986) * Fix leak on full-text index queries with syntax error (#985) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Fix leak on index scan IN optimization (#984) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Minor updates to bulk deletion (#994) * Remove redundant clone from bulk insertion of string props (#993) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Fix memory leaks on RDB-loaded strings (#991) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add flow test to validate (#967) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Update outdated logic for aggregate groups (#968) * Update outdated logic for aggregate groups * Update group.c * Update op_aggregate.c * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added all_node_scan clone (#960) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added op argument clone (#959) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added cartesian product clone (#963) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added expand into clone (#973) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added project op clone (#980) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added skip op clone (#983) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added op unwind clone (#988) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added limit op clone (#976) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added proc call clone (#979) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added filter op clone (#974) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added delete op clone (#970) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added distinct op clone (#972) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * add hash join op clone (#989) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added semi apply clone (#982) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added results op clone (#981) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added join op clone (#975) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added apply multiplexer clone (#961) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added cond var len traverse clone (#966) * added cond var len traverse clone * added conditional traverse clone * fixed PR comments * Reintroduce logic for freeing memory on run-time errors (#992) * Reintroduce logic for freeing memory on run-time errors * Remove VolatileRecord logic for freeing after run-time errors * Fix memory leaks on run-time errors in OpProject Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Free internal edge arrays on relationship matrix deletion (#997) * Free internal edge arrays on relationship matrix deletion * Update graph.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add debug function to print query (#995) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added clone for create, merge, merge_create, update (#969) * added clone for create, merge, merge_create, update * Update ast_shared.c * Update ast_shared.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Agg func fixes (#946) * Fix memory leaks in Collect function * Fix leak in aggregate func's SIValue result * Fix leak in children of aggregate function call * Simplify variable-length path free logic * Improve ownership logic in Collect, add explanatory comments * PR fix Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added label scan clone (#978) * added label scan clone * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added id seek clone (#977) * added id seek clone * added better comment on the range clone * Update op_node_by_id_seek.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added op aggreage clone (#958) * added op aggreage clone * fixed PR comments * Update op_aggregate.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Add install instructions for OpenMP (#1006) * Add install instructions for OpenMP * Update README * added op sort clone (#987) * added op sort clone * fixed PR comments * removed free_list logic * Update execution_plan.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * re-enable graph.profile (#1004) * Re-enable GraphBLAS circle-ci cache (#1007) * Fix leak on projected heap-allocated graph entities (#996) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * GRAPH.SLOWLOG (#897) * slowlog WIP * slowlog per graph * updated docs * Add longer-running query to slowlog flow test * avoid race, log only GRAPH.QUERY Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> * Use NOP label scan if range iterator construction fails (#1001) * Use NOP label scan if range iterator construction fails * deplete iterator for invalid range * dont access op consume function directly Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * removed redisearch optimization (#1010) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Note ReplicaOf memory constraint in Rampfile (#1011) * Automate leak check (#1002) * Add make memcheck rule * Add Circle memcheck job * Don't use Docker image for automated leak checking * Enable log names by migrating flow test Env initialization * Disable invalid TCK scenarios of issue #945 * Run memory test without optimizations after packaging artifacts * Add suppressions for leaks in flush-vs-shutdown race Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Write longer-running query for slowlog test (#1012) * OPType bitmask switched to contiguous enum (#1013) * OPType bitmask switched to contiguous enum * PR fixes * Clean up op tree modification code * Add static array of all scan ops * Remove unnecessary conditional * Remove static type array for unoptimized scans * Validate arity per-command rather than globally (#1023) * Ar exp param (#990) * wip * wip * after rebase * updated libcypher parser * added AR_EXP_PARAM * fixed PR comments * fixed PR comments * fixed PR comments * fixed PR comments * Update arithmetic_expression.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Fix invalid op sequence when introducing index scans (#1028) * Fix invalid op sequence when partially replacing filters with index scans * Simplify op-freeing logic in utilizeIndices * Automate testing against enterprise v5.4.14 * push down transpose operations (#1032) * push down transpose operations * added test for transposed bi-directional edge * Update algebraic_expression_optimization.c * allow skip and limit to be parametes. (#1020) * wip * test pass * added skip limit params test * added failure test * fixed PR comments * fixed online review comments * restored traverse record cap * fixing memory leaks * fixed PR comments * changed redisearch version to 1.6.11 (#1035) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added datablock out of order ops (#1022) * added datablock out of order ops * fixed PR comments * Update oo_datablock.h * Update oo_datablock.c * fixed PR comments * fixed PR comments Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Adds versioned documentation (#1034) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * deploy on package release only (#1049) * deploy on package release only * fixed yml * changed workflows * changed workflows * fixed review comments * Optional match (#1043) * Enable TCK tests * Introduce Optional and Apply ops * Modify mock AST logic * Emit error on queries beginning with OPTIONAL MATCH * Return null on property accesses of null graph entity * Disallow OPTIONAL MATCH...MATCH queries * Fix OPTIONAL filter placement * Enable TCK tests * NULL handling for path functions * NULL handling for GraphEntity and list functions * WIP improve mock AST logic * Add flow tests * Improve AST mock logic * Error handling for SET and CREATE on null entities * Record_Get refactor * Test null handling * Minor cleanup * Add documentation * Simplify toPath null handling * Improve comments * Allow OPTIONAL MATCH as first clause * Simplify null-checking logic in create ops * Use branch of Python client for testing * PR fixes * PR fixes * Remove Record_GetScalar interface * PR fixes * PR fixes * Add demo query for OPTIONAL MATCH * Use standard Python client for automation * Emit all columns as SIValues in compact formatter * Improve flow test for null entities in first result Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * added named path example (#1048) * added named path example * fixed pr comments * Better compile-time checking for undefined variables (#1063) * Improve AST validations to capture undefined variables * Propagate errors in nested AR_EXP_Evaluate failures * Add path comparison, streamline hashing logic (#1056) * Value comparison call correctly compares paths * Fix bug in DISTINCT paths and arrays * Fix memory leak in ValueHashJoin * Remove conditionals from Record hashing logic * Improve test coverage * Simplify memory management in ValueHashJoin Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Update README.md * Update index.md * Update README.md forum (#1073) * Update README.md forum * ReduceScan and ReduceTraversal respect variable scopes (#1070) * ReduceScan and ReduceTraversal respect variable scopes * Fix source lookup * PR fixes Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * make memcheck rule prints full log for files with leaks (#1072) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Error properly on graph accesses of non-graph keys (#1069) * Error properly on graph accesses of non-graph keys * Explicitly free QueryCtx on failed delete * Update cmd_delete.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * LabelScan with child reads data correctly after reset (#1086) * Guarantee that NOT conditions are unary (#1092) * is -> ID in docs (#1090) Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Alpine variant of Dockerfile (#1087) * Calling autogen.sh fails if execute permissions are not set. Rather than do that, explicitly use "sh" to run it. * Explicitly include sys/types.h Several types (u_intN_t) are defined in this header, and on some systems not included indirectly. Therefore include the file directly. * Add an Alpine variant of the Dockerfile. * Change alpine image basis to Redis v6. Tag the image as alpine (without redis version). Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Redis 6 in builds (#1060) * changed dokerfile and ci * New installation scheme in Dockerfile * fixes 1 * Review fixes 1 Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> Co-authored-by: Rafi Einstein <rafi@redislabs.com> * Fix for RAMP failure (#1093) * Graph encoding v7 - support replica-of (#1054) * added entities threshold to config * wip * added commit flow, without meta type * new graphmeta type * wip * added decode v6 * added new encoding flow * fixed edge encoding * done decoding * added entities threshold to config * wip * added commit flow, without meta type * wip * added decode v6 * added new encoding flow * fixed edge encoding * added tags * wip * wip * tests pass * wip * moved to redis 6, solved flushall. moved to uuid keys. adeded decode context * wip * moved to redis 6 events to handle keyspace * added graph pending replication * wip * wip * changed tagging * wip * tested for redis6 and redis5 * fixed ubuntu build error. added comments * changed get redis major version location * pr comments. wip * refactor. wip * fixed PR comments * fixed memeory leak * added delete guards * PR comments, wip * removed meta context type. removed uuid from meta key name * fixed edges array encoding. added tests * added skip test for redis 5 * Simplify meta key construction * fixed PR comments * added aux fields * added module replicating error. WIP * added v4, v6 rdb decode test * fixed v4 decode memory leak * fixed PR comments * fixe PR comments * fixed PR comments * fixed PR comments * fixed PR comments * added tear down to flow test to avoid race between RLTest and RG * Fix memory leak in v4 deserialization * huge refactor: single encoder/decoder logic. created config object. create redis-server version object. May support redis 5 * wip * changed virtual keys encoding logic. wip * tested on redis5. logs added * added uuid to meta keys * removed server version checking in flow tests * changed ramp * fixed redis server validation * fixed PR comments * fixed PR comments Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Update op_node_by_id_seek.c Fix, wrong variable assignment. * Update value.c removed.boolean switch * Add suppression for erroneous leak reports after DEBUG RELOAD (#1094) * Fixed compiler typo (#1097) Paragraph on OSX build should read "CXX" instead of "CPP". * Union bugfixes (#1052) * Improve scoping rules in validating UNION queries * Bugfix in uniquing column containing both nodes and edges * Add flow tests * PR fixes * PR fixes Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * Bidirectional expand into (#1047) * wip * Remove unnecessary iterator * WIP * Fix bidirectional ExpandInto, shared edge populating logic * PR fixes * Post-rebase fixes * Fix PR comments * throw runtime-error on missing query parameters (#1100) * removed query parameters annotations (#1101) * do not propagate transpose effect when introducing a transpose operation, maintain expression structure (#1102) * changed RediSearch version to 1.8 (#1103) * changed RediSearch version to 1.8 * added flag for redisearch GC * moved env var setting to memcheck script * Update memcheck.sh * fixed misplaced env var setting * fixed bad command format * flag in circle ci instead of makefile Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> * docker file for centos (#1104) * version bump Co-authored-by: Jeffrey Lovitz <jeffrey.lovitz@gmail.com> Co-authored-by: Guy Korland <gkorland@gmail.com> Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com> Co-authored-by: Sceat <11330271+Sceat@users.noreply.github.com> Co-authored-by: Itamar Haber <itamar@redislabs.com> Co-authored-by: Ariel Shtul <ariel.shtul@redislabs.com> Co-authored-by: Timothy Rule <34501912+trulede@users.noreply.github.com> Co-authored-by: Rafi Einstein <rafi@redislabs.com> Co-authored-by: Rafi Einstein <raffapen@outlook.com> Co-authored-by: Christoph Zimmermann <40485189+chrisAtRedis@users.noreply.github.com>

* decoupled result set from execution plan * after rebase merge * fixed PR comments * Restored resulte set * fixed PR comments * fixed PR comments * fixed PR comments * Update ast.c Co-authored-by: Roi Lipman <swilly22@users.noreply.github.com>

DvirDukhan requested a review from swilly22 February 19, 2020 11:06

DvirDukhan force-pushed the decouple_RS_from_EP branch from 2c32d79 to 9ba079b Compare February 19, 2020 11:08

DvirDukhan self-assigned this Feb 19, 2020

DvirDukhan added the enhancement label Feb 19, 2020

DvirDukhan added this to In progress in RedisGraph 2.2 via automation Feb 19, 2020

DvirDukhan added 2 commits February 20, 2020 14:58

decoupled result set from execution plan

3459969

after rebase merge

aa69ca3

DvirDukhan force-pushed the decouple_RS_from_EP branch from 9ba079b to aa69ca3 Compare February 20, 2020 13:53

jeffreylovitz reviewed Feb 21, 2020

View reviewed changes

RedisGraph 2.2 automation moved this from In progress to Review in progress Feb 21, 2020

jeffreylovitz suggested changes Feb 21, 2020

View reviewed changes

DvirDukhan added 2 commits February 23, 2020 09:15

Merge branch 'master' into decouple_RS_from_EP

f08a5c3

fixed PR comments

e4665ac

DvirDukhan commented Feb 23, 2020

View reviewed changes

DvirDukhan added 2 commits February 23, 2020 14:57

Merge branch 'master' into decouple_RS_from_EP

782a9ff

Restored resulte set

36a459f

DvirDukhan added 2 commits February 23, 2020 17:52

Merge branch 'master' into decouple_RS_from_EP

abf5e9e

fixed PR comments

835b32e

DvirDukhan force-pushed the decouple_RS_from_EP branch from 1baa298 to 835b32e Compare February 24, 2020 06:44

swilly22 suggested changes Feb 24, 2020

View reviewed changes

DvirDukhan and others added 3 commits February 24, 2020 13:00

fixed PR comments

682edf1

fixed PR comments

96b7c69

Update ast.c

38e16bd

swilly22 approved these changes Feb 24, 2020

View reviewed changes

Merge branch 'master' into decouple_RS_from_EP

b111fc7

swilly22 merged commit 4c220da into master Feb 24, 2020

RedisGraph 2.2 automation moved this from Review in progress to Done Feb 24, 2020

jeffreylovitz deleted the decouple_RS_from_EP branch April 9, 2020 12:24

		@@ -36,6 +36,8 @@ void ResultSet_SetColumns(ResultSet set, const char *columns);

		int ResultSet_AddRecord(ResultSet *set, Record r);

		void ResultSet_SetColumns(ResultSet set, const char *column_names);

		@@ -96,7 +96,7 @@ void Graph_Query(void *args) {
		QueryCtx_SetResultSet(result_set);

		@@ -61,7 +61,7 @@ void Graph_Profile(void *args) {
		}

		result_set = NewResultSet(ctx, false);

decoupled result set from execution plan #929

decoupled result set from execution plan #929

Conversation

DvirDukhan commented Feb 19, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeffreylovitz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment