Design of MaterializedViewQuery

[leventov]

Currently, #5556 implements it as MaterializedViewQuery and MaterializedViewQueryQueryToolChest, i. e. in the same type framework as Query and QueryToolChest. However, MaterializedViewQuery and MaterializedViewQueryQueryToolChest are shallow and pass-through classes. MaterializedViewQueryQueryToolChest couldn't sensibly return something from getResultTypeReference(), so I wonder why it didn't throw NPE in DirectDruidClient before, but I hit this problem in my change #6898.

There should probably be an abstraction like "query modifier" and other parts of the infrastructure should recognize it.

cc @zhangxinyu1 @jihoonson

[sascha-coenen]

I would propose to model materialized views as a new type of datasource instead of requiring a json wrapper around existing query types as is the current solution.
So instead of

{ 
  "queryType": "view", 
  "query": { 
    "queryType": "groupBy", 
    "dataSource": "wikiticker", 
    "granularity": "all"
    ...
  }
}

I would use the existing functionality of having different types of dataSource references to indicate that the query is to be executed on a view instead of a raw datasource like so:

{ 
    "type": "groupBy", 
    "dataSource": {
        "type": "view",
        "name": "wikiticker"
    }, 
    "granularity": "all"
    ...
}

Ref to druid documentation about different types of datasources: http://druid.io/docs/latest/querying/datasource.html

Basically, with this approach, a materialized view would be structurally equivalent to a union query.

When dataSource.type is set to "view", then the query would scan segments of the master table and any related cuboids and when setting the type to table, one could direct Druid to explicitly query only a particular materialized cuboid.

Another advantage would be that it would be more straightforward to add support for materialized views in client frameworks like plywood/pivot plyql/dsql because no json wrapper is needed anymore, just a new type of an existing syntactical building block

[leventov]

@sascha-coenen sounds interesting, would you like to make this a proposal?

[jihoonson]

@sascha-coenen sounds good to me too.

[zhangxinyu1]

@sascha-coenen Thanks for your suggestion, "view datasource" is more appropriate.

[github-actions]

This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.