Standalone Filter By Date

[etiennedi]

Background

One of the supported dataTypes in Weaviate is date, which takes an RFC3339-formatted date string.

It should be possible to use the typical compare operators (Equal, LessThan, etc.) on dates, however, that's currently not implemented yet in Standalone.

Example:

The following query shows the error message that one would encounter:

To reproduce this, you can start with a blank weaviate and import the following schema and data:

# import schema
curl localhost:8080/v1/schema/things -H 'content-type:application/json' \
  -d '{"class":"Event", "properties":[{"name": "name", "dataType": ["string"]}, {"name": "date", "dataType":["date"]}]}'

# import one data object
curl localhost:8080/v1/things -H 'content-type: application/json' \
  -d '{"class":"Event", "schema": {"name": "My Event", "date": "1985-04-12T23:20:50.52Z"}}'

# example graphql query which fails
curl localhost:8080/v1/graphql -H 'content-type: application/json' \
  -d '{"query":"{ Get{ Things{ Event(where:{ operator:LessThan path: [\"date\"] valueDate:\"2014-04-12T23:20:50.52Z\" }){ date name } } } }"}'

Goals

• operator Equal returns objects where the date matches exactly
• operator LessThan returns objects where the date is older than the query date
• operator LessThanEqual returns objects where the date is older than or equal the query date
• respectively for GreaterThan and GreaterThanEqual

Todos

tbd

Tech Overview

• The inverted index allows for range queries on bytes. For example on an int query, we store the property value as a Lexicographically Sortable Int []byte, meaning that a "less than 5" query would affect all rows from the beginning up until the row that stores 5. We can use the same for dates. I think there are two different ways we could approach this:
  • Convert the date to a timestamp (possibly with nanosecond precision) and store as an int.
  • Directly convert the string representation to a []byte which should in itself be lexicographically sortable. Possible Disadvantages: We'd need to normalize for the timezone, otherwise it's hard to tell in the inverted index if a date is really smaller/larger.
• There are two (currently removed) tests already prepared for this (copy/pasted from the previous Elasticsearch-based implementation): https://github.com/semi-technologies/weaviate/blob/00747bf2a51fe97433c760cb946ccb3125ce09b9/adapters/repos/db/filters_integration_test.go#L188-L198 It might be a good idea to add more tests (one for each operator).

[etiennedi]

Closed in #1276 - I think the PR was missing the link to this issue so it would have been auto-closed?