You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Jul 31, 2023. It is now read-only.
There are some notable inefficiencies in the internal representations for query state. Several naive representations were experimented with, all of which had the potential to dominate execution time far beyond IO costs. However, there exist well-defined solutions that could be implemented to drastically reduce overall runtime of the query engine under many or most circumstances.
In particular, there are two elements of state that need improvement:
The manner in which variable bindings are stored eagerly evaluates all cross products, requiring repeated linear traversals over potentially tens or hundreds of thousands of immutable maps.
The manner in which the query engine tracks whether or not it has already evaluated a particular predicate.
There are some notable inefficiencies in the internal representations for query state. Several naive representations were experimented with, all of which had the potential to dominate execution time far beyond IO costs. However, there exist well-defined solutions that could be implemented to drastically reduce overall runtime of the query engine under many or most circumstances.
In particular, there are two elements of state that need improvement:
To address (1), we intend to use or build a lazy relational algebra system - https://github.com/Workiva/lazy-tables
To address (2), we intend to explore feasible alternatives and choose one/more to implement.
The expectation is that we could drastically improve the query engine performance and possibly simplify several code paths.
The text was updated successfully, but these errors were encountered: