add support for filtering missing or empty data

[junaidzm13]

Feature Request

Description of Problem:

Currently, VUU doesn't have a way to filter for/out missing data. Missing data could be anything from value being null to "" (for string data type).

Preliminary analysis shows that it would be quite tricky on the server as well as the UI to support filtering of null values with our current filter operators. For example, if we try supporting null for = filter, then on the server side the parsing is relatively straightforward, but how would typeahead work for example? Similarly on the UI for example we expect numeric filters to have only valid numeric values, which means re-working our current components to be able to work with numeric as well as null or some special string representation of null? Also, how would we support null values for = filter and not allow it for other operators where is doesn't make sense e.g. order comparison > or <?

Potential Solutions:

One of the potential solutions could be to introduce new filter operator(s) to support filtering of missing or empty data.

For example, we can have two operators is and is not, with possible enum based values MISSING and EMPTY which are then converted to the relevant underlying values null and "" on the server side before applying filters.

Reference: this is also how SQL supports searching for NULL values for example: WHERE column_x IS NULL.

[chrisjstevo]

I'm would check the tableDef on the table itself for whether the column conceptually exists.

Need to check when we do this tho, might be computationally expensive to do on every row.get call

[junaidzm13]

This is more about missing data i.e. row having a value null or "" for a certain valid column. Currently, we don't have a way to filter for those. For instance, a user wants to see all the rows where price is not missing, for now we can't do this in Vuu.

[naleeha]

Need to also look at support for the null or other concepts of missing data in a calculated column
Currently its not part of parser syntax