Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add null check to mongo filtertransformer for KNOWN/UNKNOWN filters #279

Merged
merged 3 commits into from May 29, 2020

Conversation

ml-evs
Copy link
Member

@ml-evs ml-evs commented May 26, 2020

Closes #254, by adding comparison against null during KNOWN/UNKNOWN filters.

This had to be added as another post-processor, due to the mismatch between Lark/mongo. There is also some additional logic to handle the case of e.g. NOT x is UNKNOWN. This was achieved by adding a dummy magic keyword "#known" when building the lark query, which is then expanded out by the post-processor after the NOT (expr) rule has been applied.

@ml-evs ml-evs added the transformer/MongoDB Related to filter transformer for MongoDB label May 26, 2020
@ml-evs ml-evs force-pushed the ml-evs/mongo_is_known branch 2 times, most recently from 7caecde to 15d48e4 Compare May 26, 2020 18:55
@codecov
Copy link

codecov bot commented May 26, 2020

Codecov Report

Merging #279 into master will increase coverage by 0.09%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #279      +/-   ##
==========================================
+ Coverage   89.94%   90.03%   +0.09%     
==========================================
  Files          54       54              
  Lines        2248     2269      +21     
==========================================
+ Hits         2022     2043      +21     
  Misses        226      226              
Flag Coverage Δ
#unittests 90.03% <100.00%> (+0.09%) ⬆️
Impacted Files Coverage Δ
optimade/filtertransformers/mongo.py 97.02% <100.00%> (+0.34%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53a22f8...d696d42. Read the comment docs.

@ml-evs ml-evs changed the title Add type check to mongo filtertransformer for KNOWN/UNKNOWN filters Add null check to mongo filtertransformer for KNOWN/UNKNOWN filters May 27, 2020
@ml-evs ml-evs requested review from CasperWA, fekad and shyamd May 27, 2020 13:52
shyamd
shyamd previously approved these changes May 27, 2020
Copy link
Contributor

@shyamd shyamd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Copy link
Member

@CasperWA CasperWA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the use of all these postprocesses you've set up, cheers @ml-evs !

I am having some trouble wrapping my head around the logical consequences of what should be the result of each of these. I guess I should read the spec more carefully on these points.

optimade/filtertransformers/mongo.py Outdated Show resolved Hide resolved
optimade/filtertransformers/mongo.py Show resolved Hide resolved
optimade/filtertransformers/mongo.py Outdated Show resolved Hide resolved
tests/filtertransformers/test_mongo.py Outdated Show resolved Hide resolved
ml-evs and others added 2 commits May 29, 2020 16:54
Co-authored-by: CasperWelzel Andersen <CasperWA@users.noreply.github.com>
@ml-evs ml-evs requested a review from CasperWA May 29, 2020 16:02
Copy link
Member

@CasperWA CasperWA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's quite a verbose edition of the transformer now, but with these things, it's for the better I think.

Cheers @ml-evs

@CasperWA CasperWA merged commit e945a84 into master May 29, 2020
@CasperWA CasperWA deleted the ml-evs/mongo_is_known branch May 29, 2020 21:56
@ml-evs
Copy link
Member Author

ml-evs commented May 29, 2020

It's quite a verbose edition of the transformer now, but with these things, it's for the better I think.

Cheers @ml-evs

Yeah, there's no other way to handle these edge cases with just lark, unless we can get to the field names directly (I gave up trying to do that a while ago). Hopefully this is the last one!

@CasperWA
Copy link
Member

I think that would be a grave mistake, and if that is needed we have done something very wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
transformer/MongoDB Related to filter transformer for MongoDB
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Handling of KNOWN in mongo backend
3 participants