New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"NOT" predicate also excludes NULL values #244
Comments
Had a look into how that is translated into Elasticsearch and Hive queries and found this:
There's an easy but not complete fix, and it is to check if the parent predicate is a NotPredicate add the IS NOT NULL, but I guess that more complex and nested predicates need a different transformation |
upgrading to gbif-api 0.146
We should probably change Hive to use the ES-like behaviour (X != 'Y' OR X IS NULL) but if there are edge cases we should consider these first. Whatever we do, it will be important to update the documentation. |
This is rather counter intuitive. For example, in this download: https://www.gbif.org/occurrence/download/0233791-200613084148143, the user wanted to exclude invasive species:
But I don't think he realises that this also excluded all the records for which the establishment mean isn't provided.
The NOT predicate should also return NULL values (https://stackoverflow.com/questions/5658457/not-equal-operator-on-null):
establishmentMeans != 'INVASIVE' OR establishmentMeans IS NULL
The text was updated successfully, but these errors were encountered: