Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Simplifying Essence and construction of Druid query. #169
Right now, Essence is very complex. It holds current state of visualisation and provides methods to manipulate it. Because this state is represented as partial Plywood expressions, these methods are complex. Plywood expressions are not meant to be queried early and manipulated. This complexity trickles down to components that manipulate state of visualisation, like filter menus.
General idea is to keep state as POJsO with higher level semantics that Plywood expression. Semantics should correspond with semantics of application. This object also should be easy to manipulate and should be minimal. For example limit for split should be simple value, sort for split is just product of column-id and enumeration fo directions. If filters properties are derived from split definition in some way, this should be resolved lazily during query generation and not in state.
Second component should be collection of domain specific methods to manipulate state. They should purely operate on data in state. For example changing split limit would be changing one number value. Because state is minimal, these functions also should touch smallest possible part of state.
And third, function that transforms such state to Plywood expression and in consequence Druid query. All derivations should be done here and should be done in efficient way. All declarations of columns (
Additional component for this is mechanism for smart corrections to visualisation definitions. Rules could be simplified.
Main benefits are:
General road map should be like this:
After this we should have default value of visualisation state working and it should query Druid. Next step is to rewrite UI components that manipulate state to accept simpler data and functions to manipulate it.
Then we should look into possibility to write new url converter, but writing Plywood expression -> new state converters could be hard and not worth it.
Lastly we can look into manifests and rules and try to simplyfy.
ViewDefinition should be an important part of new essence POJsOs. ViewDefinition has been designed to be fully independent from Plywood expression, it defines all view properties important for external Turnilo state (URL). It should also simplified ViewDefinitionConverters (or even get rid of the converters). We should discuss all properties of the new essence not expressed in the ViewDefinition. In general most of the UI state should be bookmarkable.
+1 for rewrite clicker methods as functions, it would be nice to have an interfaces for clickers method instead of stringly typed programming using anonymous objects with functions.
+1 for makeQuery function, new essence must not be aware of Druid queries. This function should be easy to test, where input is a new essence and the output plywood expression.
Currently, we have ViewDefinition object that's created from url hash. It is POJsO and is used as seed for Essence value alongside DataCube. We could use ViewDefinition directly augmenting with some functions for actions like changing filter and updating splits accordingly. Components like SplitMenu should act on this ViewDefinition. For better API we could wrap ViewDefinition in Immutable.Record.
Using ViewDefinition as Essence
First step should be to transfer most of interfaces from ViewDefinition to Essence. Native structures should be replaced with appropriate structures from immutbale-js.
Only current version of hash/view definitions should handle