Support for Subqueries in Tremor-Query

[mfelsche]

Describe the problem you are trying to solve

Currently tremor supports composition through composing pipelines together, through function composition and through allowing references to query operator definitions and constants in externalizable modules that can be loaded via a module path.

Describe the solution you'd like

It would be excellent if the modularity in tremor extended fully to the query language so that distinct subgraphs could be modularized and consumed by multiple queries to optimise for reuse of flow oriented logic in tremor.

This would require extending module support in the tremor query language to support sub-graph definitions with parameters that can be declared and used as part of a higher level query.

Modules in tremor-query in their current state: https://docs.tremor.rs/tremor-query/modules/

A Tremor Query constitutes a DAG of nodes and their connections through which events flow on some path, decided at runtime. Subqueries also consitute a DAG and will be part of the query DAG they are used within. The challenge here is how to express such a subquery and how to integrate it into the tremor-query syntax (e.g. how to use it from a select query). Also look at how scripts and operators are defined in tremor-query.

The related LFX Summer Mentorship Project to submit applications.