Improve support for timestamp queries in MongoTransformer #590
Conversation
ml-evs
added
enhancement
Codecov Report
@@ Coverage Diff @@
## master #590 +/- ##
==========================================
- Coverage 93.27% 93.18% -0.09%
==========================================
Files 61 61
Lines 3344 3361 +17
==========================================
+ Hits 3119 3132 +13
- Misses 225 229 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
ml-evs
force-pushed
the
ml-evs/mongo_datetime
branch
3 times, most recently
from
0dcb4d3
to
a4a5530
Compare
| """ Find cases where the query dict is operating on a timestamp field. """ | ||
| if self.mapper is not None: | ||
| prop = self.mapper.alias_of(prop) | ||
| return prop == "last_modified" |
There was a problem hiding this comment.
Just a heads up, eventually this can use the mapper/schema mechanism to find all implementation-defined fields of type DataType.TIMESTAMP, but that will have to wait
ml-evs
force-pushed
the
ml-evs/mongo_datetime
branch
2 times, most recently
from
3193e0c
to
a4a5530
Compare
ml-evs
force-pushed
the
ml-evs/mongo_datetime
branch
from
a4a5530
to
1c15e7f
Compare
|
I think this was uncontroversial after our discussion on Slack a few weeks ago, so I'll merge these changes later today unless anyone stops me. |
ml-evs
force-pushed
the
ml-evs/mongo_datetime
branch
6 times, most recently
from
6f48fe0
to
9e4bfec
Compare
…ed rather than BadRequest
ml-evs
force-pushed
the
ml-evs/mongo_datetime
branch
from
9e4bfec
to
2170328
Compare
Forgot about this PR, @shyamd would be good if you could have a look as this will affect MP |
Addresses #102, where it was decided that timestamps should be handled by the transformers, rather than in the grammar themselves (which would have involved a horrible regexp, and goes slightly against the specification).
This PR adds an extra step in the
MongoTransformerto detect OPTIMADE timestamp fields (justlast_modifiedat the moment), and encode the filter value properly with BSON.Pending the next few PRs (e.g. #504), filter transformers will get access to schemas directly, allowing them to directly query for e.g. fields of type
DataType.TIMESTAMP, rather than hard-codinglast_modified.As RFC3339 does not support fractional seconds, the only operation that can be tested by the
optimade-validatoris>=(as fractional seconds will always lead to rounded times in the underlying database). This means that there will always be an inconsistency between e.g. the MongoDB server (which has milliseconds) and the JSON response (which does not).